Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimation

Understand

This paper presents our approach for the VA (Valence-Arousal) estimation task in the ABAW6 competition.

• We devised a comprehensive model by preprocessing video frames and audio segments to extract visual and audio features.
• Through the utilization of Temporal Convolutional Network (TCN) modules, we effectively captured the temporal and spatial correlations between these features.
• Subsequently, we employed a Transformer encoder structure to learn long-range dependencies, thereby enhancing the model's performance and generalization ability.