Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation

Understand

While accurate lip synchronization has been achieved for arbitrary-subject audio-driven talking face generation, the problem of how to efficiently drive the head pose remains.

• Previous methods rely on pre-estimated structural information such as landmarks and 3D parameters, aiming to generate personalized rhythmic movements.
• However, the inaccuracy of such estimated information under extreme conditions would lead to degradation problems.
• In this paper, we propose a clean yet effective framework to generate pose-controllable talking faces.