Bag of Tricks for Adversarial Training

Understand

Adversarial training (AT) is one of the most effective strategies for promoting model robustness.

• However, recent benchmarks show that most of the proposed improvements on AT are less effective than simply early stopping the training procedure.
• This counter-intuitive fact motivates us to investigate the implementation details of tens of AT methods.
• Surprisingly, we find that the basic settings (e.g., weight decay, training schedule, etc.) used in these methods are highly inconsistent.