Learning Mixtures of Gaussians Using the DDPM Objective

Understand

Recent works have shown that diffusion models can learn essentially any distribution provided one can perform score estimation.

• Yet it remains poorly understood under what settings score estimation is possible, let alone when practical gradient-based algorithms for this task can provably succeed.
• In this work, we give the first provably efficient results along these lines for one of the most fundamental distribution families, Gaussian mixture models.
• We prove that gradient descent on the denoising diffusion probabilistic model (DDPM) objective can efficiently recover the ground truth parameters of the mixture model in the following two settings: 1) We show gradient descent with random initialization learns mixtures of two spherical Gaussians in $d$ dimensions with $1/\text{poly}(d)$-separated centers.