Talking-head Generation with Rhythmic Head Motion

Understand

When people deliver a speech, they naturally move heads, and this rhythmic head motion conveys prosodic information.

• However, generating a lip-synced video while moving head naturally is challenging.
• While remarkably successful, existing works either generate still talkingface videos or rely on landmark/video frames as sparse/dense mapping guidance to generate head movements, which leads to unrealistic or uncontrollable video synthesis.
• To overcome the limitations, we propose a 3D-aware generative network along with a hybrid embedding module and a non-linear composition module.