Self-Supervised Multimodal Learning: A Survey

Understand

Multimodal learning, which aims to understand and analyze information from multiple modalities, has achieved substantial progress in the supervised regime in recent years.

• However, the heavy dependence on data paired with expensive human annotations impedes scaling up models.
• Meanwhile, given the availability of large-scale unannotated data in the wild, self-supervised learning has become an attractive strategy to alleviate the annotation bottleneck.
• Building on these two directions, self-supervised multimodal learning (SSML) provides ways to learn from raw multimodal data.