Sparta: Spatially Attentive and Adversarially Robust Activation

Understand

Adversarial training (AT) is one of the most effective ways for improving the robustness of deep convolution neural networks (CNNs).

• Just like common network training, the effectiveness of AT relies on the design of basic network components.
• In this paper, we conduct an in-depth study on the role of the basic ReLU activation component in AT for robust CNNs.
• We find that the spatially-shared and input-independent properties of ReLU activation make CNNs less robust to white-box adversarial attacks with either standard or adversarial training.