PQPP: A Joint Benchmark for Text-to-Image Prompt and Query Performance Prediction

Understand

Text-to-image generation has recently emerged as a viable alternative to text-to-image retrieval, driven by the visually impressive results of generative diffusion models.

• Although query performance prediction is an active research topic in information retrieval, to the best of our knowledge, there is no prior study that analyzes the difficulty of queries (referred to as prompts) in text-to-image generation, based on human judgments.
• To this end, we introduce the first dataset of prompts which are manually annotated in terms of image generation performance.
• Additionally, we extend these evaluations to text-to-image retrieval by collecting manual annotations that represent retrieval performance.