Convergence of Adversarial Training in Overparametrized Neural Networks

Understand

Neural networks are vulnerable to adversarial examples, i.e.

• inputs that are imperceptibly perturbed from natural data and yet incorrectly classified by the network.
• Adversarial training, a heuristic form of robust optimization that alternates between minimization and maximization steps, has proven to be among the most successful methods to train networks to be robust against a pre-defined family of perturbations.
• This paper provides a partial answer to the success of adversarial training, by showing that it converges to a network where the surrogate loss with respect to the the attack algorithm is within $\epsilon$ of the optimal robust loss.