Natural Adversarial Examples

Understand

We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade.

• The datasets are collected with a simple adversarial filtration technique to create datasets with limited spurious cues.
• Our datasets' real-world, unmodified examples transfer to various unseen models reliably, demonstrating that computer vision models have shared weaknesses.
• The first dataset is called ImageNet-A and is like the ImageNet test set, but it is far more challenging for existing models.