MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection

Understand

Fusing LiDAR and camera information is essential for achieving accurate and reliable 3D object detection in autonomous driving systems.

• This is challenging due to the difficulty of combining multi-granularity geometric and semantic features from two drastically different modalities.
• Recent approaches aim at exploring the semantic densities of camera features through lifting points in 2D camera images (referred to as seeds) into 3D space, and then incorporate 2D semantics via cross-modal interaction or fusion techniques.
• However, depth information is under-investigated in these approaches when lifting points into 3D space, thus 2D semantics can not be reliably fused with 3D points.