Towards Automatic Speech to Sign Language Generation

Understand

We aim to solve the highly challenging task of generating continuous sign language videos solely from speech segments for the first time.

• Recent efforts in this space have focused on generating such videos from human-annotated text transcripts without considering other modalities.
• However, replacing speech with sign language proves to be a practical solution while communicating with people suffering from hearing loss.
• Therefore, we eliminate the need of using text as input and design techniques that work for more natural, continuous, freely uttered speech covering an extensive vocabulary.