Fetching the paper…
Reading the bibliography…
In a series of papers \cite{LSJR16, PP17, LPP}, it was established that some of the most commonly used first order methods almost surely (under random initializations) and with step-size being small enough, avoid strict saddle points, as long as the objective function $f$ is $C^2$ and has Lipschitz gradient.
Global Stability of Dynamical Systems
M. Shub · 1987
Earlier work this paper cites.
Nonconvergence to unstable points in urn models and stochastic approximations
Robin Pemantle · 1990
Earlier work this paper cites.
Differential Equations and Dynamical Systems
Lawrence Perko · 2001
Earlier work this paper cites.
The multiplicative weights update method: a meta algorithm and applications
S. Arora, E. Hazan, and S. Kale · 2012
Earlier work this paper cites.
An extrinsic look at the riemannian hessian
Pierre-Antoine Absil, Robert Mahony, and Jochen Trumpf · 2013
Earlier work this paper cites.
Theory of convex optimization for machine learning
Sébastien Bubeck · 2014
Earlier work this paper cites.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio · 2014
Earlier work this paper cites.
Banach contraction principle and its generalizations
Abdul Latif · 2014
Earlier work this paper cites.
On the saddle point problem for non-convex optimization
Razvan Pascanu, Yann N Dauphin, Surya Ganguli, and Yoshua Bengio · 2014
Cited alongside, same era.
Understanding Machine Learning: From Theory to Algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Cited alongside, same era.
The loss surfaces of multilayer networks
Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, and Yann LeCun · 2015
Cited alongside, same era.
Escaping from saddle points; online stochastic gradient for tensor decomposition
Rong Ge, Furong Huang, Chi Jin, and Yang Yuan · 2015
Cited alongside, same era.
Global optimality of local search for low rank matrix recovery
Srinadh Bhojanapalli, Behnam Neyshabur, and Nati Srebro · 2016
Cited alongside, same era.
Matrix completion has no spurious local minimum
Multiplicative weights update with constant step-size in congestion games: Convergence, limit cycles and chaos
G. Palaiopanos, I. Panageas, and G. Piliouras · 2017
Later among the works it cites.
Gradient descent only converges to minimizers: Non-isolated critical points and invariant regions
Ioannis Panageas and Georgios Piliouras · 2017
Later among the works it cites.
Complete dictionary recovery over the sphere i: Overview and the geometric picture
Ju Sun, Qing Qu, and John Wright · 2017
Later among the works it cites.
Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method
Ju Sun, Qing Qu, and John Wright · 2017
Later among the works it cites.
Dynamical, symplectic and stochastic perspectives on gradient-based optimization
Michael I. Jordan · 2018
Later among the works it cites.
Step size matters in deep learning
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Rong Ge, Jason D Lee, and Tengyu Ma · 2016
Cited alongside, same era.
Gradient descent only converges to minimizers
Jason D Lee, Max Simchowitz, Michael I Jordan, and Benjamin Recht · 2016
Cited alongside, same era.
No spurious local minima in nonconvex low rank problems: A unified geometric analysis
Rong Ge, Chi Jin, and Yi Zheng · 2017
Cited alongside, same era.
Kamil Nar and Shankar Sastry · 2018
Later among the works it cites.
First-order methods almost always avoid saddle points
J. D. Lee, I. Panageas, G. Piliouras, M. Simchowitz, M. I. Jordan, and B. Recht · 2019
Closest in time.