HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images

Understand

The majority of learning-based semantic segmentation methods are optimized for daytime scenarios and favorable lighting conditions.

• Real-world driving scenarios, however, entail adverse environmental conditions such as nighttime illumination or glare which remain a challenge for existing approaches.
• In this work, we propose a multimodal semantic segmentation model that can be applied during daytime and nighttime.
• To this end, besides RGB images, we leverage thermal images, making our network significantly more robust.