MaxUp: A Simple Way to Improve Generalization of Neural Network Training

Understand

We propose \emph{MaxUp}, an embarrassingly simple, highly effective technique for improving the generalization performance of machine learning models, especially deep neural networks.

• The idea is to generate a set of augmented data with some random perturbations or transforms and minimize the maximum, or worst case loss over the augmented data.
• By doing so, we implicitly introduce a smoothness or robustness regularization against the random perturbations, and hence improve the generation performance.
• For example, in the case of Gaussian perturbation, \emph{MaxUp} is asymptotically equivalent to using the gradient norm of the loss as a penalty to encourage smoothness.