Spurious Feature Diversification Improves Out-of-distribution Generalization

Understand

Generalization to out-of-distribution (OOD) data is a critical challenge in machine learning.

• Ensemble-based methods, like weight space ensembles that interpolate model parameters, have been shown to achieve superior OOD performance.
• However, the underlying mechanism for their effectiveness remains unclear.
• In this study, we closely examine WiSE-FT, a popular weight space ensemble method that interpolates between a pre-trained and a fine-tuned model.