Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence

Understand

Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced performance on 2D visual tasks.

• However, improving their spatial intelligence remains a challenge.
• Existing 3D MLLMs always rely on additional 3D or 2.5D data to incorporate spatial awareness, restricting their utility in scenarios with only 2D inputs, such as images or videos.
• In this paper, we present Spatial-MLLM, a novel framework for visual-based spatial reasoning from purely 2D observations.