Understanding overfitting peaks in generalization error: Analytical risk curves for $l_2$ and $l_1$ penalized interpolation

Understanding deep learning requires rethinking generalization

Original

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 1906

Earlier work this paper cites.

Distribution of eigenvalues for some sets of random matrices

Vladimir Alexandrovich Marchenko and Leonid Andreevich Pastur · 1967

Earlier work this paper cites.

Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications

Marc Mézard, Giorgio Parisi, and Miguel Virasoro · 1987

Earlier work this paper cites.

Statistical mechanics of error-correcting codes

Yoshiyuki Kabashima and David Saad · 1999

Earlier work this paper cites.

Statistical mechanics of learning

Andreas Engel and Christian Van den Broeck · 2001

Earlier work this paper cites.

Random matrices

Madan Lal Mehta · 2004

Earlier work this paper cites.

Sparse nonnegative solution of underdetermined linear equations by linear programming

David L Donoho and Jared Tanner · 2005

Earlier work this paper cites.

Message-passing algorithms for compressed sensing

David L Donoho, Arian Maleki, and Andrea Montanari · 2009

Earlier work this paper cites.

A typical reconstruction limit for compressed sensing based on ℓ p \ell_{p} -norm minimization

Yoshiyuki Kabashima, Tadashi Wadayama, and Toshiyuki Tanaka · 2009

Earlier work this paper cites.

Information, physics, and computation

Marc Mezard and Andrea Montanari · 2009

Cited alongside, same era.

Statistical mechanics of compressed sensing

Surya Ganguli and Haim Sompolinsky · 2010

Cited alongside, same era.

Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit

David L Donoho, Yaakov Tsaig, Iddo Drori, and J-L Starck · 2012

Cited alongside, same era.

Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices

Florent Krzakala, Marc Mézard, Francois Sausset, Yifan Sun, and Lenka Zdeborová · 2012

Cited alongside, same era.

An introduction to statistical learning

Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani · 2013

Cited alongside, same era.

The cavity method for analysis of large-scale penalized regression

Overfitting or perfect fitting? risk bounds for classification and regression rules that interpolate

Original

Mikhail Belkin, Daniel Hsu, and Partha Mitra · 2018

Later among the works it cites.

To understand deep learning we need to understand kernel learning

Mikhail Belkin, Siyuan Ma, and Soumik Mandal · 2018

Later among the works it cites.

The jamming transition as a paradigm to understand the loss landscape of deep neural networks

Original

Mario Geiger, Stefano Spigler, Stéphane d’Ascoli, Levent Sagun, Marco Baity-Jesi, Giulio Biroli, and Matthieu Wyart · 2018

Later among the works it cites.

Just interpolate: Kernel" ridgeless" regression can generalize

Original

Tengyuan Liang and Alexander Rakhlin · 2018

Later among the works it cites.

Fast convergence for stochastic and distributed gradient descent in the interpolation limit