Grounded Text-to-Image Synthesis with Attention Refocusing

Understand

Driven by the scalable diffusion models trained on large-scale datasets, text-to-image synthesis methods have shown compelling results.

• However, these models still fail to precisely follow the text prompt involving multiple objects, attributes, or spatial compositions.
• In this paper, we reveal the potential causes in the diffusion model's cross-attention and self-attention layers.
• We propose two novel losses to refocus attention maps according to a given spatial layout during sampling.