Fetching the paper…

Reading the bibliography…

2016

A Lyapunov Analysis of Momentum Methods in Optimization

Wilson, Ashia C., Recht, Benjamin, Jordan, Michael I.

Original
PDF
Post on X

Understand

Momentum methods play a significant role in optimization.

Examples include Nesterov's accelerated gradient method and the conditional gradient algorithm.
Several momentum methods are provably optimal under standard oracle models, and all use a technique called estimate sequences to analyze their convergence properties.
The technique of estimate sequences has long been considered difficult to understand, leading many researchers to generate alternative, "more intuitive" methods and analyses.

A Lyapunov Analysis of Momentum Methods in Optimization · Around

Built on

B. T. Polyak
1964
Earlier work this paper cites.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams
1986
Earlier work this paper cites.
A. M. Lyapunov and A. T. Fuller
1992
Earlier work this paper cites.
J. Butcher
2000
Earlier work this paper cites.
Y. Nesterov
2004
Earlier work this paper cites.
Y. Nesterov
2005
Earlier work this paper cites.
Y. Nesterov
2008
Earlier work this paper cites.
P. Tseng
2008
Earlier work this paper cites.
Manuscript, available at http://www.optimization-online.org/DB_FILE/2009/08/2372.pdf , August 2009
M. Baes · 2009
Earlier work this paper cites.
A. Beck and M. Teboulle
2009
Earlier work this paper cites.
Y. Nesterov
2013
Earlier work this paper cites.

Similar

Y. Drori and M. Teboulle
2014
Cited alongside, same era.
Y. Nesterov
2014
Cited alongside, same era.
L. X. Qihang Lin, Zhaosong Lu
2014
Cited alongside, same era.
W. Su, S. Boyd, and E. Candes
2014
Cited alongside, same era.
S. Bubeck, Y. T. Lee, and M. Singh
Original
2015
Cited alongside, same era.
W. Krichene, A. Bayen, and P. Bartlett
2015
Cited alongside, same era.
Y. Nesterov
2015
Cited alongside, same era.

Then

D. Kim and J. A. Fessler
2016
Closest in time.
L. Lessard, B. Recht, and A. Packard
2016
Closest in time.
A. B. Taylor, J. M. Hendrickx, and F. Glineur
2016
Closest in time.
A. Wibisono, A. C. Wilson, and M. I. Jordan
Original
2016
Closest in time.
Z. Allen-Zhu
2017
Closest in time.
G. N. Grapiglia and Y. Nesterov
2017
Closest in time.
Y. Nesterov and S. U. Stich
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Y. Nesterov and V. Shikhman

2015

Cited alongside, same era.

B. O’Donoghue and E. Candès

2015

Cited alongside, same era.

Z. Allen-Zhu, P. Richtárik, Z. Qu, and Y. Yuan

2016

Cited alongside, same era.

D. Drusvyatskiy, M. Fazel, and S. Roy

2016

Cited alongside, same era.

P. L. Chebyshev

Cited in the paper.

S. Tu, S. Venkataraman, A. C. Wilson, A. Gittens, M. I. Jordan, and B. Recht

2017

Closest in time.

Arxiv preprint arXiv1802.03653, March 2018

B. Michael, J. Micheal, and W. Ashia · 2018

Closest in time.

O. Fercoq and P. Richtárik

2023

Closest in time.