The Dimpled Manifold Model of Adversarial Examples in Machine Learning

Understand

The extreme fragility of deep neural networks, when presented with tiny perturbations in their inputs, was independently discovered by several research groups in 2013.

• However, despite enormous effort, these adversarial examples remained a counterintuitive phenomenon with no simple testable explanation.
• In this paper, we introduce a new conceptual framework for how the decision boundary between classes evolves during training, which we call the {\em Dimpled Manifold Model}.
• In particular, we demonstrate that training is divided into two distinct phases.