Talking Face Generation by Adversarially Disentangled Audio-Visual Representation

Understand

Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech.

• This is a challenging task because face appearance variation and semantics of speech are coupled together in the subtle movements of the talking face regions.
• Existing works either construct specific face appearance model on specific subjects or model the transformation between lip motion and speech.
• In this work, we integrate both aspects and enable arbitrary-subject talking face generation by learning disentangled audio-visual representation.