Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures

Understand

A recent line of work has shown remarkable behaviors of the generalization error curves in simple learning models.

• Even the least-squares regression has shown atypical features such as the model-wise double descent, and further works have observed triple or multiple descents.
• Another important characteristic are the epoch-wise descent structures which emerge during training.
• The observations of model-wise and epoch-wise descents have been analytically derived in limited theoretical settings (such as the random feature model) and are otherwise experimental.