Variational Distillation of Diffusion Policies into Mixture of Experts

Understand

This work introduces Variational Diffusion Distillation (VDD), a novel method that distills denoising diffusion policies into Mixtures of Experts (MoE) through variational inference.

• Diffusion Models are the current state-of-the-art in generative modeling due to their exceptional ability to accurately learn and represent complex, multi-modal distributions.
• This ability allows Diffusion Models to replicate the inherent diversity in human behavior, making them the preferred models in behavior learning such as Learning from Human Demonstrations (LfD).
• However, diffusion models come with some drawbacks, including the intractability of likelihoods and long inference times due to their iterative sampling process.