Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision

Understand

One of the primary challenges limiting the applicability of deep learning is its susceptibility to learning spurious correlations rather than the underlying mechanisms of the task of interest.

• The resulting failure to generalise cannot be addressed by simply using more data from the same distribution.
• We propose an auxiliary training objective that improves the generalization capabilities of neural networks by leveraging an overlooked supervisory signal found in existing datasets.
• We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task.