SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces

Understand

Speech-driven 3D face animation technique, extending its applications to various multimedia fields.

• Previous research has generated promising realistic lip movements and facial expressions from audio signals.
• However, traditional regression models solely driven by data face several essential problems, such as difficulties in accessing precise labels and domain gaps between different modalities, leading to unsatisfactory results lacking precision and coherence.
• To enhance the visual accuracy of generated lip movement while reducing the dependence on labeled data, we propose a novel framework SelfTalk, by involving self-supervision in a cross-modals network system to learn 3D talking faces.