4D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understanding

Understand

Multimodal Large Language Models (MLLMs) have demonstrated impressive 2D image/video understanding capabilities.

• However, there are no publicly standardized benchmarks to assess the abilities of MLLMs in understanding the 4D objects (3D objects with temporal evolution over time).
• In this paper, we introduce 4D-Bench, the first benchmark to evaluate the capabilities of MLLMs in 4D object understanding, featuring tasks in 4D object Question Answering (4D object QA) and 4D object captioning.
• 4D-Bench provides 4D objects with diverse categories, high-quality annotations, and tasks necessitating multi-view spatial-temporal understanding, different from existing 2D image/video-based benchmarks.