DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learning

Understand

Diffusion models excel at generative modeling (e.g., text-to-image) but sampling requires multiple denoising network passes, limiting practicality.

• Efforts such as progressive distillation or consistency distillation have shown promise by reducing the number of passes at the expense of quality of the generated samples.
• In this work we identify co-variate shift as one of reason for poor performance of multi-step distilled models from compounding error at inference time.
• To address co-variate shift, we formulate diffusion distillation within imitation learning (DDIL) framework and enhance training distribution for distilling diffusion models on both data distribution (forward diffusion) and student induced distributions (backward diffusion).