On Convergence and Stability of GANs

Understand

We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions.

• We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens.
• We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse.
• We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points.