Behavior of Accelerated Gradient Methods Near Critical Points of Nonconvex Functions

A method for unconstrained convex problem with the rate of convergence O ⁡ ( 1 / k 2 ) O(1/k^{2})

Y. Nesterov · 1983

Earlier work this paper cites.

Introduction to Optimization

B. T. Polyak · 1987

Earlier work this paper cites.

Global stability of dynamical systems

M. Shub · 1987

Earlier work this paper cites.

Heavy-ball method in nonconvex optimization problems

Original

S. K. Zavriev and F. V. Kostyuk · 1993

Earlier work this paper cites.

The heavy ball with friction method, I. The continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system

H. Attouch, X. Goudou, and P. Redont · 2000

Earlier work this paper cites.

Introductory Lectures on Convex Optimization: A Basic Course

Y. Nesterov · 2004

Earlier work this paper cites.

On accelerated proximal gradient methods for convex-concave optimization

P. Tseng · 2008

Cited alongside, same era.

Convex optimization: Algorithms and complexity

S. Bubeck · 2015

Cited alongside, same era.

On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”

A. Chambolle and Ch. Dossal · 2015

Cited alongside, same era.

Accelerated proximal gradient methods for nonconvex programming

Huan Li and Zhouchen Lin · 2015

Cited alongside, same era.

The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than 1 / k 2 1/k^{2}

H. Attouch and J. Peypouquet · 2016

Cited alongside, same era.

Accelerated gradient methods for nonconvex nonlinear and stochastic programming

S. Ghadimi and G. Lan · 2016

Gradient descent only converges to minimizers

J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht · 2016

Later among the works it cites.

Gradient descent can take exponential time to escape saddle points

Simon S Du, Chi Jin, Jason D Lee, Michael I Jordan, Aarti Singh, and Barnabas Poczos · 2017

Closest in time.

Accelerated gradient descent escapes saddle points faster than gradient descent

Original

C. Jin, P. Netrapalli, and M. I. Jordan · 2017

Closest in time.

Nonlinear Optimization for Machine Learning, 2017

B. Recht and S. J. Wright · 2017

Closest in time.

Convergence rates of inertial forward-backward algorithms

H. Attouch and A. Cabot · 2018

Closest in time.