Local Convolutions Cause an Implicit Bias towards High Frequency Adversarial Examples

Understand

Adversarial Attacks are still a significant challenge for neural networks.

• Recent work has shown that adversarial perturbations typically contain high-frequency features, but the root cause of this phenomenon remains unknown.
• Inspired by theoretical work on linear full-width convolutional models, we hypothesize that the local (i.e.
• bounded-width) convolutional operations commonly used in current neural networks are implicitly biased to learn high frequency features, and that this is one of the root causes of high frequency adversarial examples.