Adversarial Training with Stochastic Weight Average

Understand

Adversarial training deep neural networks often experience serious overfitting problem.

• Recently, it is explained that the overfitting happens because the sample complexity of training data is insufficient to generalize robustness.
• In traditional machine learning, one way to relieve overfitting from the lack of data is to use ensemble methods.
• However, adversarial training multiple networks is extremely expensive.