Fetching the paper…
Reading the bibliography…
We show that accelerated gradient descent, averaged gradient descent and the heavy-ball method for non-strongly-convex problems may be reformulated as constant parameter second-order difference equation algorithms, where stability of the system is equivalent to convergence at rate O(1/n 2), where n is the number of iterations.
Some methods of speeding up the convergence of iteration methods
B. T. Polyak · 1964
Earlier work this paper cites.
A method of solving a convex programming problem with convergence rate O ( 1 / k 2 ) O(1/k^{2})
Y. Nesterov · 1983
Earlier work this paper cites.
Introduction to Optimization
B. T. Polyak · 1987
Earlier work this paper cites.
Acceleration of stochastic approximation by averaging
B. T. Polyak and A. B. Juditsky · 1992
Earlier work this paper cites.
Random dynamical systems
L. Arnold · 1998
Earlier work this paper cites.
Iterative solution of nonlinear equations in several variables , volume 30 of Classics in Applied Mathematics
J. M. Ortega and W. C. Rheinboldt · 2000
Earlier work this paper cites.
Optimal rates of aggregation
A. B. Tsybakov · 2003
Earlier work this paper cites.
Introductory Lectures on Convex Optimization , volume 87 of Applied Optimization
Y. Nesterov · 2004
Earlier work this paper cites.
Smooth optimization with approximate gradient
A. d’Aspremont · 2008
Cited alongside, same era.
A fast iterative shrinkage-thresholding algorithm for linear inverse problems
A. Beck and M. Teboulle · 2009
Cited alongside, same era.
Accelerated gradient methods for stochastic optimization and online learning
C. Hu, W. Pan, and J. T. Kwok · 2009
Cited alongside, same era.
Dual averaging methods for regularized stochastic learning and online optimization
L. Xiao · 2010
Cited alongside, same era.
Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning
F. Bach and E. Moulines · 2011
Cited alongside, same era.
Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization
M. Schmidt, N. Le Roux, and F. Bach · 2011
Cited alongside, same era.
Non-strongly-convex smooth stochastic approximation with convergence rate O ( 1 / n ) O(1/n)
F. Bach and E. Moulines · 2013
Later among the works it cites.
Gradient methods for minimizing composite functions
Y. Nesterov · 2013
Later among the works it cites.
Adaptive restart for accelerated gradient schemes
B. O’Donoghue and E. Candes · 2013
Later among the works it cites.
Constant step size least-mean-square: Bias-variance trade-offs and optimal sampling distributions
A. Défossez and F. Bach · 2014
Later among the works it cites.
First-order methods of smooth convex optimization with inexact oracle
O. Devolder, F. Glineur, and Y. Nesterov · 2014
Later among the works it cites.
Non-parametric Stochastic Approximation with Large Step sizes
A. Dieuleveut and F. Bach · 2014
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization
A. Agarwal, P. L. Bartlett, P. Ravikumar, and M. J. Wainwright · 2012
Cited alongside, same era.
An optimal method for stochastic composite optimization
G. Lan · 2012
Cited alongside, same era.
A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights
W. Su, S. Boyd, and E. Candes · 2014
Later among the works it cites.