Text-based Editing of Talking-head Video

Understand

Editing talking-head video to change the speech content or to remove filler words is challenging.

• We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e.
• no jump cuts).
• Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame.