A-Bench: Are LMMs Masters at Evaluating AI-generated Images?

Understand

How to accurately and efficiently assess AI-generated images (AIGIs) remains a critical challenge for generative models.

• Given the high costs and extensive time commitments required for user studies, many researchers have turned towards employing large multi-modal models (LMMs) as AIGI evaluators, the precision and validity of which are still questionable.
• Furthermore, traditional benchmarks often utilize mostly natural-captured content rather than AIGIs to test the abilities of LMMs, leading to a noticeable gap for AIGIs.
• Therefore, we introduce A-Bench in this paper, a benchmark designed to diagnose whether LMMs are masters at evaluating AIGIs.