Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance

Understand

The synthesis of 3D facial animations from speech has garnered considerable attention.

• Due to the scarcity of high-quality 4D facial data and well-annotated abundant multi-modality labels, previous methods often suffer from limited realism and a lack of lexible conditioning.
• We address this challenge through a trilogy.
• We first introduce Generalized Neural Parametric Facial Asset (GNPFA), an efficient variational auto-encoder mapping facial geometry and images to a highly generalized expression latent space, decoupling expressions and identities.