MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences

Understand

Human communication is multimodal in nature; it is through multiple modalities such as language, voice, and facial expressions, that opinions and emotions are expressed.

• Data in this domain exhibits complex multi-relational and temporal interactions.
• Learning from this data is a fundamentally challenging research problem.
• In this paper, we propose Modal-Temporal Attention Graph (MTAG).