Lipschitz Singularities in Diffusion Models

Understand

Diffusion models, which employ stochastic differential equations to sample images through integrals, have emerged as a dominant class of generative models.

• However, the rationality of the diffusion process itself receives limited attention, leaving the question of whether the problem is well-posed and well-conditioned.
• In this paper, we explore a perplexing tendency of diffusion models: they often display the infinite Lipschitz property of the network with respect to time variable near the zero point.
• We provide theoretical proofs to illustrate the presence of infinite Lipschitz constants and empirical results to confirm it.