Speech Driven Talking Face Generation from a Single Image and an Emotion Condition

Understand

Visual emotion expression plays an important role in audiovisual speech communication.

• In this work, we propose a novel approach to rendering visual emotion expression in speech-driven talking face generation.
• Specifically, we design an end-to-end talking face generation system that takes a speech utterance, a single face image, and a categorical emotion label as input to render a talking face video synchronized with the speech and expressing the conditioned emotion.
• Objective evaluation on image quality, audiovisual synchronization, and visual emotion expression shows that the proposed system outperforms a state-of-the-art baseline system.