Fetching the paper…
Reading the bibliography…
Given a non-convex twice differentiable cost function f, we prove that the set of initial conditions so that gradient descent converges to saddle points where \nabla^2 f has at least one strictly negative eigenvalue has (Lebesgue) measure zero, even for cost functions f with non-isolated critical points, answering an open question in [Lee, Simchowitz, Jordan, Recht, COLT2016].
General Topology
John L. Kelley · 1955
Earlier work this paper cites.
Calculus On Manifolds: A Modern Approach To Classical Theorems Of Advanced Calculus
Michael Spivak · 1965
Earlier work this paper cites.
On the use of directions of negative curvature in a modified newton method
Jorge J Moré and Danny C Sorensen · 1979
Earlier work this paper cites.
Global Stability of Dynamical Systems
Michael Shub · 1987
Earlier work this paper cites.
Nonconvergence to unstable points in urn models and stochastic approximations
Robin Pemantle · 1990
Earlier work this paper cites.
Differential Equations and Dynamical Systems
Lawrence Perko · 1991
Earlier work this paper cites.
Trust region methods
Andrew R Conn, Nicholas IM Gould, and Ph L Toint · 2000
Earlier work this paper cites.
Algorithms for non-negative matrix factorization
Daniel D Lee and H Sebastian Seung · 2001
Earlier work this paper cites.
Introductory lectures on convex optimization
Yurii Nesterov · 2004
Earlier work this paper cites.
Cubic regularization of newton method and its global performance
Yurii Nesterov and Boris T Polyak · 2006
Earlier work this paper cites.
Engineering optimization: methods and applications
A Ravindran, Gintaras Victor Reklaitis, and Kenneth Martin Ragsdell · 2006
Cited alongside, same era.
Matrix completion from a few entries
Raghunandan H Keshavan, Sewoong Oh, and Andrea Montanari · 2009
Cited alongside, same era.
Multiplicative updates outperform generic no-regret learning in congestion games
Robert Kleinberg, Georgios Piliouras, and Eva Tardos · 2009
Cited alongside, same era.
Evolutionary game theory
William H Sandholm · 2009
Cited alongside, same era.
Algorithms, games, and evolution
Erick Chastain, Adi Livnat, Christos Papadimitriou, and Umesh Vazirani · 2014
Cited alongside, same era.
The loss surfaces of multilayer networks
Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, and Yann LeCun · 2014
Simple, efficient, and neural algorithms for sparse coding
Sanjeev Arora, Rong Ge, Tengyu Ma, and Ankur Moitra · 2015
Later among the works it cites.
Phase retrieval via wirtinger flow: Theory and algorithms
Emmanuel J Candes, Xiaodong Li, and Mahdi Soltanolkotabi · 2015
Later among the works it cites.
Escaping from saddle points—online stochastic gradient for tensor decomposition
Rong Ge, Furong Huang, Chi Jin, and Yang Yuan · 2015
Later among the works it cites.
Natural selection as an inhibitor of genetic diversity: Multiplicative weights updates algorithm and a conjecture of haploid genetics
Ruta Mehta, Ioannis Panageas, and Georgios Piliouras · 2015
Later among the works it cites.
Mutation, Sexual Reproduction and Survival in Dynamic Environments
Ruta Mehta, Ioannis Panageas, Georgios Piliouras, Prasad Tetali, and Vijay V. Vazirani · 2015
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio · 2014
Cited alongside, same era.
Ruta Mehta, Ioannis Panageas, Georgios Piliouras, and Sadra Yazdanbod · 2014
Cited alongside, same era.
Spectral methods meet em: A provably optimal algorithm for crowdsourcing
Yuchen Zhang, Xi Chen, Denny Zhou, and Michael I Jordan · 2014
Cited alongside, same era.
Later among the works it cites.
On sex, evolution, and the multiplicative weights update algorithm
Reshef Meir and David Parkes · 2015
Later among the works it cites.
Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method
Ju Sun, Qing Qu, and John Wright · 2015
Later among the works it cites.
Gradient descent only converges to minimizers
Jason D. Lee, Max Simchowitz, Michael I. Jordan, and Benjamin Recht · 2016
Closest in time.
Ioannis Panageas and Georgios Piliouras · 2016
Closest in time.