Fetching the paper…
Reading the bibliography…
Stochastic gradient descent (SGD) for strongly convex functions converges at the rate $\bO(1/k)$.
A stochastic approximation method
Herbert Robbins and Sutton Monro · 1951
Earlier work this paper cites.
A new approach to linear filtering and prediction problems
Rudolph Emil Kalman · 1960
Earlier work this paper cites.
Some methods of speeding up the convergence of iteration methods
Boris T Polyak · 1964
Earlier work this paper cites.
Introduction to optimization. translations series in mathematics and engineering
Boris T Polyak · 1987
Earlier work this paper cites.
Variable target value subgradient method
Sehun Kim, Hyunsil Ahn, and Seong-Cheol Cho · 1990
Earlier work this paper cites.
Stochastic gradient learning in neural networks
Léon Bottou · 1991
Earlier work this paper cites.
Information-theoretic lower bounds on the oracle complexity of convex optimization
Alekh Agarwal, Martin J Wainwright, Peter L Bartlett, and Pradeep K Ravikumar · 2009
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer · 2011
Earlier work this paper cites.
Information-based complexity, feedback and dynamics in convex programming
Maxim Raginsky and Alexander Rakhlin · 2011
Earlier work this paper cites.
Making gradient descent optimal for strongly convex stochastic optimization
Alexander Rakhlin, Ohad Shamir, and Karthik Sridharan · 2011
Earlier work this paper cites.
Rmsprop: Divide the gradient by a running average of its recent magnitude
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky · 2012
Cited alongside, same era.
Simon Lacoste-Julien, Mark Schmidt, and Francis Bach · 2012
Cited alongside, same era.
Minimization methods for non-differentiable functions
Naum Zuselevich Shor · 2012
Cited alongside, same era.
Adadelta: an adaptive learning rate method
Matthew D Zeiler · 2012
Cited alongside, same era.
Subgradient methods
Stephen Boyd and Almir Mutapcic · 2013
Cited alongside, same era.
Accelerating stochastic gradient descent using predictive variance reduction
Train faster, generalize better: Stability of stochastic gradient descent
Moritz Hardt, Benjamin Recht, and Yoram Singer · 2015
Later among the works it cites.
Optimization methods for large-scale machine learning
Léon Bottou, Frank E Curtis, and Jorge Nocedal · 2016
Later among the works it cites.
First-Order Methods in Optimization
Amir Beck · 2017
Later among the works it cites.
The marginal value of adaptive gradient methods in machine learning
Ashia C Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht · 2017
Later among the works it cites.
On the insufficiency of existing momentum schemes for stochastic optimization
Rahul Kidambi, Praneeth Netrapalli, Prateek Jain, and Sham Kakade · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Rie Johnson and Tong Zhang · 2013
Cited alongside, same era.
Introductory lectures on convex optimization: A basic course
Yurii Nesterov · 2013
Cited alongside, same era.
Linear convergence with condition number independent access of full gradients
Lijun Zhang, Mehrdad Mahdavi, and Rong Jin · 2013
Cited alongside, same era.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Cited alongside, same era.
Striving for simplicity: The all convolutional net
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller · 2014
Cited alongside, same era.
On the convergence of adam and beyond
Sashank J Reddi, Satyen Kale, and Sanjiv Kumar · 2018
Later among the works it cites.
L4: Practical loss-based stepsize adaptation for deep learning
Michal Rolinek and Georg Martius · 2018
Later among the works it cites.
Wngrad: learn the learning rate in gradient descent
Xiaoxia Wu, Rachel Ward, and Léon Bottou · 2018
Later among the works it cites.
Sgd with arbitrary sampling: General analysis and improved rates
Xun Qian, Peter Richtarik, Robert Gower, Alibek Sailanbayev, Nicolas Loizou, and Egor Shulgin · 2019
Closest in time.