DETR4D: Direct Multi-View 3D Object Detection with Sparse Attention

Understand

3D object detection with surround-view images is an essential task for autonomous driving.

• In this work, we propose DETR4D, a Transformer-based framework that explores sparse attention and direct feature query for 3D object detection in multi-view images.
• We design a novel projective cross-attention mechanism for query-image interaction to address the limitations of existing methods in terms of geometric cue exploitation and information loss for cross-view objects.
• In addition, we introduce a heatmap generation technique that bridges 3D and 2D spaces efficiently via query initialization.