Unleashing Text-to-Image Diffusion Models for Visual Perception

Understand

Diffusion models (DMs) have become the new trend of generative models and have demonstrated a powerful ability of conditional synthesis.

• Among those, text-to-image diffusion models pre-trained on large-scale image-text pairs are highly controllable by customizable prompts.
• Unlike the unconditional generative models that focus on low-level attributes and details, text-to-image diffusion models contain more high-level knowledge thanks to the vision-language pre-training.
• In this paper, we propose VPD (Visual Perception with a pre-trained Diffusion model), a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks.