Fetching the paper…
Reading the bibliography…
We demonstrate the ability of hybrid regularization methods to automatically avoid the double descent phenomenon arising in the training of random feature models (RFM).
Two models of double descent for weak features
Mikhail Belkin, Daniel Hsu, and Ji Xu · 1903
Earlier work this paper cites.
Calculating the singular values and pseudo-inverse of a matrix
Gene Golub and William Kahan · 1965
Earlier work this paper cites.
Generalized cross-validation as a method for choosing a good ridge parameter
Gene H Golub, Michael Heath, and Grace Wahba · 1979
Earlier work this paper cites.
Handwritten digit recognition with a back-propagation network
Y LeCun, B E Boser, and J S Denker · 1990
Earlier work this paper cites.
An implicit shift bidiagonalization algorithm for ill-posed systems
Åke Björck, Eric Grimme, and Paul Van Dooren · 1994
Earlier work this paper cites.
A study of cross-validation and bootstrap for accuracy estimation and model selection
Ron Kohavi et al · 1995
Earlier work this paper cites.
Regularization of inverse problems , volume 375
Heinz Werner Engl, Martin Hanke, and Andreas Neubauer · 1996
Earlier work this paper cites.
Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion
Per Christian Hansen · 1998
Earlier work this paper cites.
Estimation of the l-curve via lanczos bidiagonalization
Daniela Calvetti, Gene Howard Golub, and Lothar Reichel · 1999
Earlier work this paper cites.
Computational methods for inverse problems
Curtis R Vogel · 2002
Earlier work this paper cites.
Random features for large-scale kernel machines
Ali Rahimi and Benjamin Recht · 2007
Cited alongside, same era.
A weighted gcv method for lanczos hybrid regularization
Julianne Chung, James G Nagy, and Dianne P O’leary · 2008
Cited alongside, same era.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Cited alongside, same era.
Discrete inverse problems , volume 7 of Fundamentals of Algorithms
Per Christian Hansen · 2010
Cited alongside, same era.
On early stopping in gradient descent learning
Yuan Yao, Lorenzo Rosasco, and Andrea Caponnetto · 2012
Cited alongside, same era.
A hybrid lsmr algorithm for large-scale tikhonov regularization
Julianne Chung and Katrina Palmer · 2015
IR Tools: a MATLAB package of iterative regularization methods and large-scale test problems
Silvia Gazzola, Per Christian Hansen, and James G Nagy · 2018
Later among the works it cites.
A model of double descent for high-dimensional binary linear classification
Zeyu Deng, Abla Kammoun, and Christos Thrampoulidis · 2019
Later among the works it cites.
Ir tools: a matlab package of iterative regularization methods and large-scale test problems
Silvia Gazzola, Per Christian Hansen, and James G Nagy · 2019
Later among the works it cites.
Surprises in high-dimensional ridgeless least squares interpolation
Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J Tibshirani · 2019
Later among the works it cites.
The generalization error of random features regression: Precise asymptotics and double descent curve
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
On krylov projection methods and tikhonov regularization
Silvia Gazzola, Paolo Novati, and Maria Rosaria Russo · 2015
Cited alongside, same era.
Deep learning , volume 1
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio · 2016
Cited alongside, same era.
Generalized hybrid iterative methods for large-scale bayesian inverse problems
Julianne Chung and Arvind K Saibaba · 2017
Cited alongside, same era.
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal
Cited in the paper.
Lsqr: An algorithm for sparse linear equations and sparse least squares
Christopher C Paige and Michael A Saunders
Cited in the paper.
Algorithm 583: Lsqr: Sparse linear equations and least squares problems
Christopher C Paige and Michael A Saunders
Cited in the paper.
Song Mei and Andrea Montanari · 2019
Later among the works it cites.
High-dimensional dynamics of generalization error in neural networks
Madhu S Advani, Andrew M Saxe, and Haim Sompolinsky · 2020
Closest in time.
Scaling description of generalization with number of parameters in deep learning
Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, Stéphane d’Ascoli, Giulio Biroli, Clément Hongler, and Matthieu Wyart · 2020
Closest in time.
Analytic study of double descent in binary classification: The impact of loss
Ganesh Kini and Christos Thrampoulidis · 2020
Closest in time.
The Slow Deterioration of the Generalization Error of the Random Feature Model
Chao Ma, Lei Wu, and Weinan E · 2020
Closest in time.