Multi-Modal 3D Object Detection by Box Matching

Understand

Multi-modal 3D object detection has received growing attention as the information from different sensors like LiDAR and cameras are complementary.

• Most fusion methods for 3D detection rely on an accurate alignment and calibration between 3D point clouds and RGB images.
• However, such an assumption is not reliable in a real-world self-driving system, as the alignment between different modalities is easily affected by asynchronous sensors and disturbed sensor placement.
• We propose a novel {F}usion network by {B}ox {M}atching (FBMNet) for multi-modal 3D detection, which provides an alternative way for cross-modal feature alignment by learning the correspondence at the bounding box level to free up the dependency of calibration during inference.