Text-driven Visual Synthesis with Latent Diffusion Prior

Understand

There has been tremendous progress in large-scale text-to-image synthesis driven by diffusion models enabling versatile downstream applications such as 3D object synthesis from texts, image editing, and customized generation.

• We present a generic approach using latent diffusion models as powerful image priors for various visual synthesis tasks.
• Existing methods that utilize such priors fail to use these models' full capabilities.
• To improve this, our core ideas are 1) a feature matching loss between features from different layers of the decoder to provide detailed guidance and 2) a KL divergence loss to regularize the predicted latent features and stabilize the training.