Progressive Distillation for Fast Sampling of Diffusion Models

Understand

Diffusion models have recently shown great promise for generative modeling, outperforming GANs on perceptual quality and autoregressive models at density estimation.

• A remaining downside is their slow sampling time: generating high quality samples takes many hundreds or thousands of model evaluations.
• Here we make two contributions to help eliminate this downside: First, we present new parameterizations of diffusion models that provide increased stability when using few sampling steps.
• Second, we present a method to distill a trained deterministic diffusion sampler, using many steps, into a new diffusion model that takes half as many sampling steps.