Avoiding spurious correlations via logit correction

Understand

Empirical studies suggest that machine learning models trained with empirical risk minimization (ERM) often rely on attributes that may be spuriously correlated with the class labels.

• Such models typically lead to poor performance during inference for data lacking such correlations.
• In this work, we explicitly consider a situation where potential spurious correlations are present in the majority of training data.
• In contrast with existing approaches, which use the ERM model outputs to detect the samples without spurious correlations and either heuristically upweight or upsample those samples, we propose the logit correction (LC) loss, a simple yet effective improvement on the softmax cross-entropy loss, to correct the sample logit.