Removing Undesirable Feature Contributions Using Out-of-Distribution Data

Understand

Several data augmentation methods deploy unlabeled-in-distribution (UID) data to bridge the gap between the training and inference of neural networks.

• However, these methods have clear limitations in terms of availability of UID data and dependence of algorithms on pseudo-labels.
• Herein, we propose a data augmentation method to improve generalization in both adversarial and standard learning by using out-of-distribution (OOD) data that are devoid of the abovementioned issues.
• We show how to improve generalization theoretically using OOD data in each learning scenario and complement our theoretical analysis with experiments on CIFAR-10, CIFAR-100, and a subset of ImageNet.