Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation

Understand

Personalized text-to-image generation methods can generate customized images based on the reference images, which have garnered wide research interest.

• Recent methods propose a finetuning-free approach with a decoupled cross-attention mechanism to generate personalized images requiring no test-time finetuning.
• However, when multiple reference images are provided, the current decoupled cross-attention mechanism encounters the object confusion problem and fails to map each reference image to its corresponding object, thereby seriously limiting its scope of application.
• To address the object confusion problem, in this work we investigate the relevance of different positions of the latent image features to the target object in diffusion model, and accordingly propose a weighted-merge method to merge multiple reference image features into the corresponding objects.