Fetching the paper…
Reading the bibliography…
We derive high-dimensional scaling limits and fluctuations for the online least-squares Stochastic Gradient Descent (SGD) algorithm by taking the properties of the data generating model explicitly into consideration.
A stochastic approximation method
H. Robbins and S. Monro · 1951
Earlier work this paper cites.
Efficient estimations from a slowly convergent Robbins-Monro process
D. Ruppert · 1988
Earlier work this paper cites.
Ordinary differential equations
V. I. Arnold · 1992
Earlier work this paper cites.
Acceleration of stochastic approximation by averaging
B. T. Polyak and A. B. Juditsky · 1992
Earlier work this paper cites.
Stochastic differential equations in infinite-dimensional spaces
G. Kallianpur and J. Xiong · 1995
Earlier work this paper cites.
Stochastic Burgers and KPZ equations from particle systems
L. Bertini and G. Giacomin · 1997
Earlier work this paper cites.
Scaling limits of interacting particle systems , volume 320
C. Kipnis and C. Landim · 1998
Earlier work this paper cites.
Stochastic Interacting Systems: Contact, Voter and Exclusion Processes , volume 324
T. Liggett · 1999
Earlier work this paper cites.
Continuous martingales and Brownian motion , volume 293 of Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]
D. Revuz and M. Yor · 1999
Earlier work this paper cites.
Sobolev spaces
R. A. Adams and J. J. Fournier · 2003
Earlier work this paper cites.
Stochastic approximation and recursive algorithms and applications , volume 35
H. J. Kushner and G. George Yin · 2003
Earlier work this paper cites.
Topics in random walks in random environment
A.-S. Sznitman · 2004
Earlier work this paper cites.
Random walks in random environment
O. Zeitouni · 2004
Earlier work this paper cites.
Toeplitz and circulant matrices: A review
R. M. Gray et al · 2006
Earlier work this paper cites.
Random fields and geometry , volume 80
R. J. Adler and J. E. Taylor · 2007
Earlier work this paper cites.
Optimal rates for the regularized least-squares algorithm
A. Caponnetto and E. De Vito · 2007
Earlier work this paper cites.
Differential equation approximations for Markov chains
R. Darling and J. Norris · 2008
Earlier work this paper cites.
Convergence of probability measures
P. Billingsley · 2009
Earlier work this paper cites.
Order of current variance and diffusivity in the asymmetric simple exclusion process
M. Balázs and T. Seppäläinen · 2010
Earlier work this paper cites.
Introduction to KPZ
J. Quastel · 2011
Earlier work this paper cites.
Adaptive algorithms and stochastic approximations , volume 22
A. Benveniste, M. Métivier, and P. Priouret · 2012
Earlier work this paper cites.
The Kardar–Parisi–Zhang equation and universality class
I. Corwin · 2012
Earlier work this paper cites.
Brownian Motion and Stochastic Calculus , volume 113
I. Karatzas and S. Shreve · 2012
Earlier work this paper cites.
Stochastic approximation methods for constrained and unconstrained systems , volume 26
H. J. Kushner and D. S. Clark · 2012
Earlier work this paper cites.
Stochastic approximation and optimization of random systems , volume 17
L. Ljung, G. Pflug, and H. Walk · 2012
Earlier work this paper cites.
Diffusion limits of the Random Walk Metropolis algorithm in high dimensions
J. C. Mattingly, N. S. Pillai, and A. M. Stuart · 2012
Earlier work this paper cites.
Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions
N. S. Pillai, A. M. Stuart, and A. H. Thiéry · 2012
Earlier work this paper cites.
Large scale dynamics of interacting particles
H. Spohn · 2012
Earlier work this paper cites.
A generalized Isserlis theorem for location mixtures of Gaussian random vectors
C. Vignat · 2012
Earlier work this paper cites.
Stochastic equations in infinite dimensions
G. Da Prato and J. Zabczyk · 2014
Earlier work this paper cites.
An introduction to computational stochastic PDEs , volume 50
G. J. Lord, C. E. Powell, and T. Shardlow · 2014
Earlier work this paper cites.
Harder, better, faster, stronger convergence rates for least-squares regression
A. Dieuleveut, N. Flammarion, and F. Bach · 2017
Earlier work this paper cites.
Hall-Littlewood-PushTASEP and its KPZ limit
P. Ghosal · 2017
Cited alongside, same era.
Acceleration and averaging in stochastic descent dynamics
W. Krichene and P. L. Bartlett · 2017
Cited alongside, same era.
Optimal rates for multi-pass stochastic gradient methods
J. Lin and L. Rosasco · 2017
Cited alongside, same era.
Asymptotic and finite-sample properties of estimators based on stochastic gradients
P. Toulis and E. M. Airoldi · 2017
Cited alongside, same era.
C. Wang, J. Mattingly, and Y. M. Lu · 2017
Cited alongside, same era.
Near-optimal inference in adaptive linear regression
K. Khamaru, Y. Deshpande, T. Lattimore, L. Mackey, and M. J. Wainwright · 2021
Later among the works it cites.
The KPZ fixed point
K. Matetski, J. Quastel, and D. Remenik · 2021
Later among the works it cites.
Dynamics of stochastic momentum methods on large-scale, quadratic models
C. Paquette and E. Paquette · 2021
Later among the works it cites.
SGD in the large: Average-case analysis, asymptotics, and stepsize criticality
C. Paquette, K. Lee, F. Pedregosa, and E. Paquette · 2021
Later among the works it cites.
Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field training
H. T. Pham and P.-M. Nguyen · 2021
Later among the works it cites.
Almost sure convergence rates for stochastic gradient descent and stochastic heavy ball
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
L. Bottou, F. E. Curtis, and J. Nocedal · 2018
Cited alongside, same era.
On the global convergence of gradient descent for over-parameterized models using optimal transport
L. Chizat and F. Bach · 2018
Cited alongside, same era.
ASEP ( q , j ) {\rm ASEP}(q,j) converges to the KPZ equation
I. Corwin, H. Shen, and L.-C. Tsai · 2018
Cited alongside, same era.
Online bootstrap confidence intervals for the stochastic gradient descent estimator
Y. Fang, J. Xu, and L. Yang · 2018
Cited alongside, same era.
Near-optimal stochastic approximation for online principal component estimation
C. J. Li, M. Wang, H. Liu, and T. Zhang · 2018
Cited alongside, same era.
A mean field view of the landscape of two-layer neural networks
S. Mei, A. Montanari, and P.-M. Nguyen · 2018
Cited alongside, same era.
Statistical optimality of stochastic gradient descent on hard learning problems through multiple passes
L. Pillaud-Vivien, A. Rudi, and F. Bach · 2018
Cited alongside, same era.
O. Sebbouh, R. M. Gower, and A. Defazio · 2021
Later among the works it cites.
An analysis of constant step size SGD in the non-convex regime: Asymptotic normality and bias
L. Yu, K. Balasubramanian, S. Volgushev, and M. A. Erdogdu · 2021
Later among the works it cites.
Expressions for joint moments of elliptical distributions
B. Zuo, C. Yin, and N. Balakrishnan · 2021
Later among the works it cites.
The merged-staircase property: A necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks
E. Abbe, E. B. Adsera, and T. Misiakiewicz · 2022
Later among the works it cites.
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation
J. Ba, M. A. Erdogdu, T. Suzuki, Z. Wang, D. Wu, and G. Yang · 2022
Later among the works it cites.
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling
G. Ben Arous, R. Gheissari, and A. Jagannath · 2022
Later among the works it cites.
K. A. Chandrasekher, M. Lou, and A. Pananjady · 2022
Later among the works it cites.
The directed landscape
D. Dauvergne, J. Ortmann, and B. Virág · 2022
Later among the works it cites.
The conjugate gradient algorithm on a general class of spiked covariance matrices
X. Ding and T. Trogdon · 2022
Later among the works it cites.
Overparameterization of deep ResNet: Zero loss and mean-field analysis
Z. Ding, S. Chen, Q. Li, and S. J. Wright · 2022
Later among the works it cites.
A. Durmus, E. Moulines, A. Naumov, and S. Samsonov · 2022
Later among the works it cites.
Rigorous dynamical mean field theory for stochastic gradient descent methods
C. Gerbelot, E. Troiani, F. Mignacco, F. Krzakala, and L. Zdeborova · 2022
Later among the works it cites.
Conservative SPDEs as fluctuating mean field limits of stochastic gradient descent
B. Gess, R. S. Gvalani, and V. Konarovskyi · 2022
Later among the works it cites.
On almost sure convergence rates of stochastic gradient methods
J. Liu and Y. Yuan · 2022
Later among the works it cites.
Control systems and reinforcement learning
S. Meyn · 2022
Later among the works it cites.
Variational inference in high-dimensional linear regression
S. Mukherjee and S. Sen · 2022
Later among the works it cites.
The tap free energy for high-dimensional linear regression
J. Qiu and S. Sen · 2022
Later among the works it cites.
Trainability and accuracy of artificial neural networks: An interacting particle system approach
G. Rotskoff and E. Vanden-Eijnden · 2022
Later among the works it cites.
Berry–Esseen bounds for multivariate nonlinear statistics with applications to M-estimators and stochastic gradient descent algorithms
Q.-M. Shao and Z.-S. Zhang · 2022
Later among the works it cites.
Mean field analysis of deep neural networks
J. Sirignano and K. Spiliopoulos · 2022
Later among the works it cites.
Phase diagram of stochastic gradient descent in high-dimensional two-layer neural networks
R. Veiga, L. Stephan, B. Loureiro, F. Krzakala, and L. Zdeborová · 2022
Later among the works it cites.
High-dimensional Central Limit Theorems for Linear Functionals of Online Least-Squares SGD
B. Agrawalla, K. Balasubramanian, and P. Ghosal · 2023
Closest in time.
Sharp global convergence guarantees for iterative nonconvex optimization with random data
K. A. Chandrasekher, A. Pananjady, and C. Thrampoulidis · 2023
Closest in time.
Asymptotic normality and optimality in nonsmooth stochastic approximation
D. Davis, D. Drusvyatskiy, and L. Jiang · 2023
Closest in time.
Convergence of exclusion processes and the KPZ equation to the KPZ fixed point
J. Quastel and S. Sarkar · 2023
Closest in time.
Online stochastic gradient descent with arbitrary initialization solves non-smooth, non-convex phase retrieval
Y. S. Tan and R. Vershynin · 2023
Closest in time.