Speech2Video: Cross-Modal Distillation for Speech to Video Generation

Understand

This paper investigates a novel task of talking face video generation solely from speeches.

• The speech-to-video generation technique can spark interesting applications in entertainment, customer service, and human-computer-interaction industries.
• Indeed, the timbre, accent and speed in speeches could contain rich information relevant to speakers' appearance.
• The challenge mainly lies in disentangling the distinct visual attributes from audio signals.