Can Implicit Bias Imply Adversarial Robustness?

Understand

The implicit bias of gradient-based training algorithms has been considered mostly beneficial as it leads to trained networks that often generalize well.

• However, Frei et al.
• (2023) show that such implicit bias can harm adversarial robustness.
• Specifically, they show that if the data consists of clusters with small inter-cluster correlation, a shallow (two-layer) ReLU network trained by gradient flow generalizes well, but it is not robust to adversarial attacks of small radius.