The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study

Understand

3D Large Language Models (LLMs) leveraging spatial information in point clouds for 3D spatial reasoning attract great attention.

• Despite some promising results, the advantages of point clouds over other modalities remain unclear.
• Moreover, existing 3D benchmarks are insufficient for fairly evaluating the ability of multimodal LLMs to comprehend spatial concepts.
• To address these challenges, we introduce ScanReQA, a 3D spatial reasoning benchmark encompassing text, vision, and point cloud modalities.