Fetching the paper…
Reading the bibliography…
We develop the mathematical foundations of the stochastic modified equations (SME) framework for analyzing the dynamics of stochastic gradient algorithms, where the latter is approximated by a class of stochastic differential equations with small noise parameters.
Difference methods and the equations of hydrodynamics
WF Noh and MH Protter · 1960
Earlier work this paper cites.
The stability properties of a coupled pair of non-linear partial difference equations
Bart J Daly · 1963
Earlier work this paper cites.
Heuristic stability theory for finite-difference equations
CW Hirt · 1968
Earlier work this paper cites.
The pricing of options and corporate liabilities
Fischer Black and Myron Scholes · 1973
Earlier work this paper cites.
The modified equation approach to the stability and accuracy analysis of finite-difference methods
RF Warming and BJ Hyett · 1974
Earlier work this paper cites.
Approximate integration of stochastic differential equations
Grigori N Milstein · 1975
Earlier work this paper cites.
Rates of convergence for sequential monte carlo optimization methods
Harold J Kushner · 1978
Earlier work this paper cites.
A method for solving the convex programming problem with convergence rate 𝒪 ( 1 / k 2 ) \mathcal{O}(1/k^{2})
Yurii E Nesterov · 1983
Earlier work this paper cites.
An invariant measure approach to the convergence of stochastic approximations with state dependent noise
Harold J Kushner and Adam Shwartz · 1984
Earlier work this paper cites.
Weak approximation of solutions of systems of stochastic differential equations
Grigori N Milstein · 1986
Earlier work this paper cites.
Stochastic approximation and recursive algorithms and applications , volume 35
Harold Kushner and G George Yin · 2003
Earlier work this paper cites.
Probability: theory and examples
Rick Durrett · 2010
Earlier work this paper cites.
Partial differential equations
Lawrence C Evans · 2010
Cited alongside, same era.
Numerical Solution of Stochastic Differential Equations
Peter E. Kloeden and Eckhard Platen · 2011
Cited alongside, same era.
Non-asymptotic analysis of stochastic approximation algorithms for machine learning
Eric Moulines and Francis Bach · 2011
Cited alongside, same era.
Stochastic approximation methods for constrained and unconstrained systems , volume 26
Harold Joseph Kushner and Dean S Clark · 2012
Cited alongside, same era.
Stochastic approximation and optimization of random systems , volume 17
Lennart Ljung, Georg Ch Pflug, and Harro Walk · 2012
Cited alongside, same era.
Non-strongly-convex smooth stochastic approximation with convergence rate 𝒪 ( 1 / n ) \mathcal{O}(1/n)
Francis Bach and Eric Moulines · 2013
A proximal stochastic gradient method with progressive variance reduction
Lin Xiao and Tong Zhang · 2014
Later among the works it cites.
Averaged least-mean-squares: Bias-variance trade-offs and optimal sampling distributions
Alexandre Défossez and Francis Bach · 2015
Later among the works it cites.
Dynamics of stochastic gradient algorithms. arxiv preprint
Qianxiao Li, Cheng Tai, and Weinan E · 2015
Later among the works it cites.
Continuous-time limit of stochastic gradient descent revisited
Stephan Mandt, Matthew D Hoffman, and David M Blei · 2015
Later among the works it cites.
A variational analysis of stochastic gradient algorithms
Stephan Mandt, Matthew D Hoffman, and David M Blei · 2016
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Introductory lectures on convex optimization: A basic course , volume 87
Yurii Nesterov · 2013
Cited alongside, same era.
Stochastic differential equations: an introduction with applications
Bernt Oksendal · 2013
Cited alongside, same era.
Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes
Ohad Shamir and Tong Zhang · 2013
Cited alongside, same era.
Stochastic gradient descent, weighted sampling, and the randomized algorithm
Deanna Needell, Rachel Ward, and Nati Srebro · 2014
Cited alongside, same era.
Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
Shai Shalev-Shwartz and Tong Zhang · 2014
Cited alongside, same era.
A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights
Weijie Su, Stephen Boyd, and Emmanuel Candes · 2014
Cited alongside, same era.
A variational perspective on accelerated methods in optimization
Andre Wibisono, Ashia C Wilson, and Michael I Jordan · 2016
Later among the works it cites.
A note on semi-groups of stochastic gradient descent and online principal component analysis
Yuanyuan Feng, Lei Li, and Jian-Guo Liu · 2017
Later among the works it cites.
On the diffusion approximation of nonconvex stochastic gradient descent
Wenqing Hu, Chris Junchi Li, Lei Li, and Jian-Guo Liu · 2017
Later among the works it cites.
Stochastic modified equations and adaptive stochastic gradient algorithms
Qianxiao Li, Cheng Tai, and Weinan E · 2017
Later among the works it cites.
Stochastic gradient descent as approximate bayesian inference
Stephan Mandt, Matthew D Hoffman, and David M Blei · 2017
Later among the works it cites.
Stochastic modified equations for the asynchronous stochastic gradient descent
Jing An, Jianfeng Lu, and Lexing Ying · 2018
Closest in time.
Michael Betancourt, Michael I Jordan, and Ashia C Wilson · 2018
Closest in time.