MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection

Understand

In this paper, we address the challenging modality-agnostic semantic segmentation (MaSS), aiming at centering the value of every modality at every feature granularity.

• Training with all available visual modalities and effectively fusing an arbitrary combination of them is essential for robust multi-modal fusion in semantic segmentation, especially in real-world scenarios, yet remains less explored to date.
• Existing approaches often place RGB at the center, treating other modalities as secondary, resulting in an asymmetric architecture.
• However, RGB alone can be limiting in scenarios like nighttime, where modalities such as event data excel.