Increasing Robustness to Spurious Correlations using Forgettable Examples

Understand

Neural NLP models tend to rely on spurious correlations between labels and input features to perform their tasks.

• Minority examples, i.e., examples that contradict the spurious correlations present in the majority of data points, have been shown to increase the out-of-distribution generalization of pre-trained language models.
• In this paper, we first propose using example forgetting to find minority examples without prior knowledge of the spurious correlations present in the dataset.
• Forgettable examples are instances either learned and then forgotten during training or never learned.