MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspective

Understand

Large Multimodal Models (LMMs) demonstrate impressive capabilities.

• However, current benchmarks predominantly focus on image comprehension in specific domains, and these benchmarks are labor-intensive to construct.
• Moreover, their answers tend to be brief, making it difficult to assess the ability of LMMs to generate detailed descriptions of images.
• To address these limitations, we propose the MMGenBench-Pipeline, a straightforward and fully automated evaluation pipeline.