ControlThinker: Unveiling Latent Semantics for Controllable Image Generation through Visual Reasoning

Understand

The field of controllable image generation has seen significant advancements, with various architectures improving generation layout consistency with control signals.

• However, contemporary methods still face challenges in bridging the semantic gap between input text prompts with sparse semantics and the target images, often over-relying on low-level control signals to infer regional details.
• To address this challenge, we propose ControlThinker, a novel framework that employs a "comprehend-then-generate" paradigm.
• Firstly, by incentivizing the visual reasoning capability of a MLLM, latent semantics from control images are mined to enrich text prompts.