Fetching the paper…
Reading the bibliography…
We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels.
Technics of field renormalization and dynamics of critical phenomena
Cirano de Dominicis · 1976
Earlier work this paper cites.
Spin glass theory and beyond
Marc Mézard, Giorgio Parisi, and Miguel A. Virasoro · 1987
Earlier work this paper cites.
Query by committee
H Sebastian Seung, Manfred Opper, and Haim Sompolinsky · 1992
Earlier work this paper cites.
New method for studying the dynamics of disordered spin systems without finite-size effects
H Eissfeller and M Opper · 1992
Earlier work this paper cites.
Supersymmetry in spin glass dynamics
Jorge Kurchan · 1992
Earlier work this paper cites.
Mean-field monte carlo approach to the sherrington-kirkpatrick model with asymmetric couplings
H Eissfeller and M Opper · 1994
Earlier work this paper cites.
Exact solution for on-line learning in multilayer neural networks
David Saad and Sara A Solla · 1995
Earlier work this paper cites.
On-line learning in soft committee machines
David Saad and Sara A Solla · 1995
Earlier work this paper cites.
Dynamical mean-field theory of strongly correlated fermion systems and the limit of infinite dimensions
Antoine Georges, Gabriel Kotliar, Werner Krauth, and Marcelo J Rozenberg · 1996
Earlier work this paper cites.
Quantum field theory and critical phenomena
Jean Zinn-Justin · 1996
Earlier work this paper cites.
Dynamics of training
Siegfried Bös and Manfred Opper · 1997
Earlier work this paper cites.
Symmetric langevin spin glass dynamics
G Ben Arous, Alice Guionnet, et al · 1997
Earlier work this paper cites.
Supersymmetry, replica and dynamic treatments of disordered systems: a parallel presentation
Jorge Kurchan · 2002
Earlier work this paper cites.
Margin maximizing loss functions
Saharon Rosset, Ji Zhu, and Trevor J Hastie · 2004
Earlier work this paper cites.
On-line learning in neural networks
David Saad · 2009
Cited alongside, same era.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M Saxe, James L McClelland, and Surya Ganguli · 2013
Cited alongside, same era.
Exponential expressivity in deep neural networks through transient chaos
Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli · 2016
Cited alongside, same era.
Spurious local minima are common in two-layer relu neural networks
Itay Safran and Ohad Shamir · 2017
Cited alongside, same era.
Deep information propagation, 2017
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein · 2017
Cited alongside, same era.
A mean field theory of batch normalization
Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, and Samuel S. Schoenholz · 2019
Later among the works it cites.
Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit, 2019
Song Mei, Theodor Misiakiewicz, and Andrea Montanari · 2019
Later among the works it cites.
Dynamical isometry and a mean field theory of lstms and grus, 2019
Dar Gilboa, Bo Chang, Minmin Chen, Greg Yang, Samuel S. Schoenholz, Ed H. Chi, and Jeffrey Pennington · 2019
Later among the works it cites.
Bayesian deep convolutional networks with many channels are gaussian processes
Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Daniel A. Abolafia, Jeffrey Pennington, and Jascha Sohl-dickstein · 2019
Later among the works it cites.
Passed & spurious: Descent algorithms and local minima in spiked matrix-tensor models
Stefano Sarao Mannelli, Florent Krzakala, Pierfrancesco Urbani, and Lenka Zdeborova · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Universality of the sat-unsat (jamming) threshold in non-convex continuous constraint satisfaction problems
Silvio Franz, Giorgio Parisi, Maxim Sevelev, Pierfrancesco Urbani, and Francesco Zamponi · 2017
Cited alongside, same era.
Grant M Rotskoff and Eric Vanden-Eijnden · 2018
Cited alongside, same era.
A mean field view of the landscape of two-layer neural networks
Song Mei, Andrea Montanari, and Phan-Minh Nguyen · 2018
Cited alongside, same era.
On the global convergence of gradient descent for over-parameterized models using optimal transport
Lenaic Chizat and Francis Bach · 2018
Cited alongside, same era.
Out-of-equilibrium dynamical mean-field equations for the perceptron model
Elisabeth Agoritsas, Giulio Biroli, Pierfrancesco Urbani, and Francesco Zamponi · 2018
Cited alongside, same era.
The implicit bias of gradient descent on separable data
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro · 2018
Cited alongside, same era.
Bad global minima exist and sgd can reach them
Shengchao Liu, Dimitris Papailiopoulos, and Dimitris Achlioptas · 2019
Cited alongside, same era.
Jamming in multilayer supervised learning models
Silvio Franz, Sungmin Hwang, and Pierfrancesco Urbani · 2019
Later among the works it cites.
A model of double descent for high-dimensional binary linear classification
Zeyu Deng, Abla Kammoun, and Christos Thrampoulidis · 2019
Later among the works it cites.
Numerical implementation of dynamical mean field theory for disordered systems: application to the lotka–volterra model of ecosystems
Felix Roy, Giulio Biroli, Guy Bunin, and Chiara Cammarota · 2019
Later among the works it cites.
Theory of Simple Glasses: Exact Solutions in Infinite Dimensions
Giorgio Parisi, Pierfrancesco Urbani, and Francesco Zamponi · 2020
Closest in time.
Mean-field inference methods for neural networks
Marylou Gabrié · 2020
Closest in time.
Marvels and pitfalls of the langevin algorithm in noisy high-dimensional inference
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, and Lenka Zdeborová · 2020
Closest in time.
The role of regularization in classification of high-dimensional noisy gaussian mixture
Francesca Mignacco, Florent Krzakala, Yue M Lu, and Lenka Zdeborová · 2020
Closest in time.
Numerical solution of the dynamical mean field theory of infinite-dimensional equilibrium liquids
Alessandro Manacorda, Grégory Schehr, and Francesco Zamponi · 2020
Closest in time.