Addressing Image Hallucination in Text-to-Image Generation through Factual Image Retrieval

Understand

Text-to-image generation has shown remarkable progress with the emergence of diffusion models.

• However, these models often generate factually inconsistent images, failing to accurately reflect the factual information and common sense conveyed by the input text prompts.
• We refer to this issue as Image hallucination.
• Drawing from studies on hallucinations in language models, we classify this problem into three types and propose a methodology that uses factual images retrieved from external sources to generate realistic images.