Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Models

Understand

Text-to-image diffusion models have shown great success in generating high-quality text-guided images.

• Yet, these models may still fail to semantically align generated images with the provided text prompts, leading to problems like incorrect attribute binding and/or catastrophic object neglect.
• Given the pervasive object-oriented structure underlying text prompts, we introduce a novel object-conditioned Energy-Based Attention Map Alignment (EBAMA) method to address the aforementioned problems.
• We show that an object-centric attribute binding loss naturally emerges by approximately maximizing the log-likelihood of a $z$-parameterized energy-based model with the help of the negative sampling technique.