AQ-GT: a Temporally Aligned and Quantized GRU-Transformer for Co-Speech Gesture Synthesis

Understand

The generation of realistic and contextually relevant co-speech gestures is a challenging yet increasingly important task in the creation of multimodal artificial agents.

• Prior methods focused on learning a direct correspondence between co-speech gesture representations and produced motions, which created seemingly natural but often unconvincing gestures during human assessment.
• We present an approach to pre-train partial gesture sequences using a generative adversarial network with a quantization pipeline.
• The resulting codebook vectors serve as both input and output in our framework, forming the basis for the generation and reconstruction of gestures.