StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentation

Understand

Multimodal semantic segmentation shows significant potential for enhancing segmentation accuracy in complex scenes.

• However, current methods often incorporate specialized feature fusion modules tailored to specific modalities, thereby restricting input flexibility and increasing the number of training parameters.
• To address these challenges, we propose StitchFusion, a straightforward yet effective modal fusion framework that integrates large-scale pre-trained models directly as encoders and feature fusers.
• This approach facilitates comprehensive multi-modal and multi-scale feature fusion, accommodating any visual modal inputs.