Exploiting Negative Curvature in Deterministic and Stochastic Optimization

An iteration method for the solution of the eigenvalue problem of linear differential and integral operators

Cornelius Lanczos · 1950

Earlier work this paper cites.

On the use of directions of negative curvature in a modified newton method

Jorge J Moré and Danny C Sorensen · 1979

Earlier work this paper cites.

Curvilinear path steplength algorithms for minimization which use directions of negative curvature

Donald Goldfarb · 1980

Earlier work this paper cites.

Computing modified newton directions using a partial cholesky factorization

Anders Forsgren, Philip E Gill, and Walter Murray · 1995

Earlier work this paper cites.

Exploiting negative curvature directions in linesearch methods for unconstrained optimization

N. I. M. Gould, S. Lucidi, M. Roma, and PH. L. Toint · 2000

Earlier work this paper cites.

A Box-Constrained Optimization Algorithm with Negative Curvature Directions and Spectral Projected Gradients

E. G. Birgin and J. M. Martínez · 2001

Earlier work this paper cites.

Numerical Optimization

J. Nocedal and S. J. Wright · 2006

Earlier work this paper cites.

Sparse subspace clustering

Ehsan Elhamifar and René Vidal · 2009

Earlier work this paper cites.

Deep learning via hessian-free optimization

James Martens · 2010

Earlier work this paper cites.

Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio · 2014

Cited alongside, same era.

Equilibrated adaptive learning rates for non-convex optimization

Yann Dauphin, Harm de Vries, and Yoshua Bengio · 2015

Cited alongside, same era.

Escaping From Saddle Points — Online Stochastic Gradient for Tensor Decomposition

R. Ge, F. Huang, C. Jin, and Y. Yuan · 2015

Cited alongside, same era.

Cutest: a constrained and unconstrained testing environment with safe threads for mathematical optimization

Nicholas IM Gould, Dominique Orban, and Philippe L Toint · 2015

Cited alongside, same era.

A solver for nonconvex bound-constrained quadratic optimization

Hassan Mohy-ud-Din and Daniel P. Robinson · 2015

Cited alongside, same era.

On large-batch training for deep learning: Generalization gap and sharp minima

Original

Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang · 2016

Later among the works it cites.

Gradient descent only converges to minimizers

Jason D. Lee, Max Simchowitz, Michael I. Jordan, and Benjamin Recht · 2016

Later among the works it cites.

A nonconvex formulation for low rank subspace clustering: Algorithms and convergence analysis

Hao Jiang, Daniel P. Robinson, and René Vidal · 2017

Closest in time.

How to escape saddle points efficiently

Original

Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M Kakade, and Michael I Jordan · 2017

Closest in time.

First-order methods almost always avoid saddle points

J. D. Lee, I. Panageas, G. Piliouras, M. Simchowitz, M. I. Jordan, and B. Recht · 2017

Closest in time.