MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models

Understand

Given the remarkable success that large visual language models (LVLMs) have achieved in image perception tasks, the endeavor to make LVLMs perceive the world like humans is drawing increasing attention.

• Current multi-modal benchmarks primarily focus on facts or specific topic-related knowledge contained within individual images.
• However, they often overlook the associative relations between multiple images, which require the identification and analysis of similarities among entities or content present in different images.
• Therefore, we propose the multi-image relation association task and a meticulously curated Multi-granularity Multi-image Relational Association (MMRA) benchmark, comprising 1,024 samples.