End-to-End Multi-View Lipreading

Understand

Non-frontal lip views contain useful information which can be used to enhance the performance of frontal view lipreading.

• However, the vast majority of recent lipreading works, including the deep learning approaches which significantly outperform traditional approaches, have focused on frontal mouth images.
• As a consequence, research on joint learning of visual features and speech classification from multiple views is limited.
• In this work, we present an end-to-end multi-view lipreading system based on Bidirectional Long-Short Memory (BLSTM) networks.