Gradient penalty from a maximum margin perspective

Understand

A popular heuristic for improved performance in Generative adversarial networks (GANs) is to use some form of gradient penalty on the discriminator.

• This gradient penalty was originally motivated by a Wasserstein distance formulation.
• However, the use of gradient penalty in other GAN formulations is not well motivated.
• We present a unifying framework of expected margin maximization and show that a wide range of gradient-penalized GANs (e.g., Wasserstein, Standard, Least-Squares, and Hinge GANs) can be derived from this framework.