Benign Overfitting in Linear Regression

Understand

The phenomenon of benign overfitting is one of the key mysteries uncovered by deep learning methodology: deep neural networks seem to predict well, even with a perfect fit to noisy training data.

• Motivated by this phenomenon, we consider when a perfect fit to training data in linear regression is compatible with accurate prediction.
• We give a characterization of linear regression problems for which the minimum norm interpolating prediction rule has near-optimal prediction accuracy.
• The characterization is in terms of two notions of the effective rank of the data covariance.