Deep Learning without Poor Local Minima

Some NP-complete problems in quadratic and nonlinear programming

Murty, Katta G, & Kabadi, Santosh N. 1987 · 1987

Earlier work this paper cites.

Linear learning: Landscapes and algorithms

Baldi, Pierre. 1989 · 1989

Earlier work this paper cites.

Neural networks and principal component analysis: Learning from examples without local minima

Baldi, Pierre, & Hornik, Kurt. 1989 · 1989

Earlier work this paper cites.

Training a 3-node neural network is NP-complete

Blum, Avrim L, & Rivest, Ronald L. 1992 · 1992

Earlier work this paper cites.

The Schur complement and its applications

Zhang, Fuzhen. 2006 · 2006

Earlier work this paper cites.

Variational analysis

Rockafellar, R Tyrrell, & Wets, Roger J-B. 2009 · 2009

Cited alongside, same era.

Complex-valued autoencoders

Baldi, Pierre, & Lu, Zhiqin. 2012 · 2012

Cited alongside, same era.

Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

Dauphin, Yann N, Pascanu, Razvan, Gulcehre, Caglar, Cho, Kyunghyun, Ganguli, Surya, & Bengio, Yoshua. 2014 · 2014

Cited alongside, same era.

On the computational efficiency of training neural networks

Livni, Roi, Shalev-Shwartz, Shai, & Shamir, Ohad. 2014 · 2014

Cited alongside, same era.

The Loss Surfaces of Multilayer Networks

Choromanska, Anna, Henaff, MIkael, Mathieu, Michael, Ben Arous, Gerard, & LeCun, Yann. 2015a

Cited in the paper.

Open Problem: The landscape of the loss surfaces of multilayer networks

Choromanska, Anna, LeCun, Yann, & Arous, Gérard Ben. 2015b

Cited in the paper.

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

Saxe, Andrew M, McClelland, James L, & Ganguli, Surya. 2014 · 2014

Later among the works it cites.

Escaping From Saddle Points—Online Stochastic Gradient for Tensor Decomposition

Ge, Rong, Huang, Furong, Jin, Chi, & Yuan, Yang. 2015 · 2015

Later among the works it cites.

Deep Learning

Goodfellow, Ian, Bengio, Yoshua, & Courville, Aaron. 2016 · 2016

Closest in time.

Learning Real and Boolean Functions: When Is Deep Better Than Shallow

Mhaskar, Hrushikesh, Liao, Qianli, & Poggio, Tomaso. 2016 · 2016

Closest in time.