Object-level Visual Prompts for Compositional Image Generation

Understand

We introduce a method for composing object-level visual prompts within a text-to-image diffusion model.

• Our approach addresses the task of generating semantically coherent compositions across diverse scenes and styles, similar to the versatility and expressiveness offered by text prompts.
• A key challenge in this task is to preserve the identity of the objects depicted in the input visual prompts, while also generating diverse compositions across different images.
• To address this challenge, we introduce a new KV-mixed cross-attention mechanism, in which keys and values are learned from distinct visual representations.