Fetching the paper…
Reading the bibliography…
Can we accelerate convergence of gradient descent without changing the algorithm -- just by carefully choosing stepsizes? Surprisingly, we show that the answer is yes.
Methods of conjugate gradients for solving linear systems
Magnus R Hestenes, Eduard Stiefel, et al · 1952
Earlier work this paper cites.
On Richardson’s method for solving linear systems with positive definite matrices
David Young · 1953
Earlier work this paper cites.
Some methods of speeding up the convergence of iteration methods
Boris T Polyak · 1964
Earlier work this paper cites.
Convex analysis
Ralph Tyrell Rockafellar · 1970
Earlier work this paper cites.
Ordering of the iterative parameters in the cyclical Chebyshev iterative method
VI Lebedev and SA Finogenov · 1971
Earlier work this paper cites.
An introduction to the approximation of functions
Theodore J Rivlin · 1981
Earlier work this paper cites.
Problem complexity and method efficiency in optimization
Arkadii Nemirovskii and David Borisovich Yudin · 1983
Earlier work this paper cites.
Linear and nonlinear programming , volume 2
David G Luenberger and Yinyu Ye · 1984
Earlier work this paper cites.
Introduction to optimization
Boris T. Polyak · 1987
Earlier work this paper cites.
Two-point step size gradient methods
Jonathan Barzilai and Jonathan M Borwein · 1988
Earlier work this paper cites.
From potential theory to matrix iterations in six steps
Tobin A Driscoll, Kim-Chuan Toh, and Lloyd N Trefethen · 1998
Earlier work this paper cites.
Introductory lectures on convex optimization: A basic course , volume 87
Yurii Nesterov · 1998
Earlier work this paper cites.
Nonlinear programming
D.P. Bertsekas · 1999
Earlier work this paper cites.
Numerical Optimization
J. Nocedal and S. J. Wright · 1999
Earlier work this paper cites.
Computational commutative algebra. 1
M. Kreuzer and L. Robbiano · 2000
Earlier work this paper cites.
Matrix Iterative Analysis , volume 27 of
Richard S Varga · 2000
Earlier work this paper cites.
Algorithms in real algebraic geometry , volume 10 of
S. Basu, R. Pollack, and M.-F. Roy · 2003
Earlier work this paper cites.
Convex optimization
Stephen Boyd and Lieven Vandenberghe · 2004
Earlier work this paper cites.
On accelerated proximal gradient methods for convex-concave optimization
Paul Tseng · 2008
Earlier work this paper cites.
A fast iterative shrinkage-thresholding algorithm for linear inverse problems
Amir Beck and Marc Teboulle · 2009
Earlier work this paper cites.
Gradient algorithms for quadratic optimization with fast convergence rates
Luc Pronzato and Anatoly Zhigljavsky · 2011
Earlier work this paper cites.
Quantifier elimination and cylindrical algebraic decomposition
Bob F Caviness and Jeremy R Johnson · 2012
Earlier work this paper cites.
Ideals, varieties, and algorithms: an introduction to computational algebraic geometry and commutative algebra
David Cox, John Little, and Donal O’Shea · 2013
Cited alongside, same era.
An asymptotically optimal gradient algorithm for quadratic optimization with low computational cost
Anatoly Zhigljavsky, Luc Pronzato, and Elena Bukina · 2013
Cited alongside, same era.
Linear coupling: An ultimate unification of gradient and mirror descent
Zeyuan Allen-Zhu and Lorenzo Orecchia · 2014
Cited alongside, same era.
First-order methods of smooth convex optimization with inexact oracle
Olivier Devolder, François Glineur, and Yurii Nesterov · 2014
Cited alongside, same era.
Performance of first-order methods for smooth convex minimization: a novel approach
Yoel Drori and Marc Teboulle · 2014
Cited alongside, same era.
Smooth strongly convex interpolation and exact worst-case performance of first-order methods
Adrien B Taylor, Julien M Hendrickx, and François Glineur · 2017
Later among the works it cites.
Greed, hedging, and acceleration in convex optimization
Jason M. Altschuler · 2018
Later among the works it cites.
Efficient first-order methods for convex minimization: a constructive approach
Yoel Drori and Adrien B Taylor · 2018
Later among the works it cites.
An optimal first order method based on optimal quadratic averaging
Dmitriy Drusvyatskiy, Maryam Fazel, and Scott Roy · 2018
Later among the works it cites.
The fastest known globally convergent first-order method for minimizing strongly convex functions
Bryan Van Scoy, Randy A Freeman, and Kevin M Lynch · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Convex optimization: Algorithms and complexity
Sébastien Bubeck · 2015
Cited alongside, same era.
A geometric alternative to Nesterov’s accelerated gradient descent
Sébastien Bubeck, Yin Tat Lee, and Mohit Singh · 2015
Cited alongside, same era.
Optimal black-box reductions between optimization objectives
Zeyuan Allen-Zhu and Elad Hazan · 2016
Cited alongside, same era.
Introduction to online convex optimization
Elad Hazan · 2016
Cited alongside, same era.
Optimized first-order methods for smooth convex minimization
Donghwan Kim and Jeffrey A Fessler · 2016
Cited alongside, same era.
Analysis and design of optimization algorithms via integral quadratic constraints
Laurent Lessard, Benjamin Recht, and Andrew Packard · 2016
Cited alongside, same era.
A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights
Weijie Su, Stephen Boyd, and Emmanuel J Candès · 2016
Cited alongside, same era.
Acceleration via symplectic discretization of high-resolution differential equations
Bin Shi, Simon S Du, Weijie Su, and Michael I Jordan · 2019
Later among the works it cites.
Super-convergence: Very fast training of neural networks using large learning rates
Leslie N Smith and Nicholay Topin · 2019
Later among the works it cites.
Relative lipschitzness in extragradient methods and a direct recipe for acceleration
Michael B Cohen, Aaron Sidford, and Kevin Tian · 2020
Later among the works it cites.
Worst-case convergence analysis of inexact gradient and Newton methods through semidefinite programming performance estimation
Etienne De Klerk, Francois Glineur, and Adrien B Taylor · 2020
Later among the works it cites.
Operator splitting performance estimation: Tight contraction factors and optimal parameter selection
Ernest K Ryu, Adrien B Taylor, Carolina Bergeling, and Pontus Giselsson · 2020
Later among the works it cites.
Acceleration via fractal learning rate schedules
Naman Agarwal, Surbhi Goel, and Cyril Zhang · 2021
Later among the works it cites.
Generalized momentum-based methods: A Hamiltonian perspective
Jelena Diakonikolas and Michael I Jordan · 2021
Later among the works it cites.
Acceleration methods
Alexandre d’Aspremont, Damien Scieur, and Adrien Taylor · 2021
Later among the works it cites.
Provable super-convergence with a large cyclical learning rate
Samet Oymak · 2021
Later among the works it cites.
Understanding the acceleration phenomenon via high-resolution differential equations
Bin Shi, Simon S Du, Michael I Jordan, and Weijie J Su · 2021
Later among the works it cites.
Shuvomoy Das Gupta, Bart PG Van Parys, and Ernest K Ryu · 2022
Later among the works it cites.
On the oracle complexity of smooth strongly convex minimization
Yoel Drori and Adrien Taylor · 2022
Later among the works it cites.
Super-acceleration with cyclical step-sizes
Baptiste Goujaud, Damien Scieur, Aymeric Dieuleveut, Adrien B Taylor, and Fabian Pedregosa · 2022
Later among the works it cites.
Principled analyses and design of first-order methods with inexact proximal operators
Mathieu Barré, Adrien B Taylor, and Francis Bach · 2023
Closest in time.
Provably faster gradient descent via long steps
Benjamin Grimmer · 2023
Closest in time.
A systematic approach to Lyapunov analyses of continuous-time models in convex optimization
Céline Moucer, Adrien B. Taylor, and Francis Bach · 2023
Closest in time.
An optimal gradient method for smooth strongly convex minimization
Adrien Taylor and Yoel Drori · 2023
Closest in time.