Robustness and Accuracy Could Be Reconcilable by (Proper) Definition

Understand

The trade-off between robustness and accuracy has been widely studied in the adversarial literature.

• Although still controversial, the prevailing view is that this trade-off is inherent, either empirically or theoretically.
• Thus, we dig for the origin of this trade-off in adversarial training and find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance -- an overcorrection towards smoothness.
• Given this, we advocate employing local equivariance to describe the ideal behavior of a robust model, leading to a self-consistent robust error named SCORE.