Differentially Private Diffusion Models

Understand

While modern machine learning models rely on increasingly large training datasets, data is often limited in privacy-sensitive domains.

• Generative models trained with differential privacy (DP) on sensitive data can sidestep this challenge, providing access to synthetic data instead.
• We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs), which enforce privacy using differentially private stochastic gradient descent (DP-SGD).
• We investigate the DM parameterization and the sampling algorithm, which turn out to be crucial ingredients in DPDMs, and propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs.