Theory II: Landscape of the Empirical Risk in Deep Learning

K. Fukushima, “Neocognitron: A self-organizing neural network for a mechanism of pattern recognition unaffected by shift in position,” Biological Cybernetics

1980

Earlier work this paper cites.

M. Shub and S. Smale, “Complexity of bezout theorem v: Polynomial time,” Theoretical Computer Science

1994

Earlier work this paper cites.

M. Riesenhuber and T. Poggio, “Hierarchical models of object recognition in cortex,” Nature Neuroscience

1999

Earlier work this paper cites.

Springer Science & Business Media, 2005

I. Borg and P. J. Groenen, Modern multidimensional scaling: Theory and applications · 2005

Earlier work this paper cites.

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167

Original

2015

Earlier work this paper cites.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature

2015

Cited alongside, same era.

A. Choromanska, Y. LeCun, and B. Arous, “Open problem: The landscape of the loss surfaces of multilayer networks,” JMLR: Workshop and Conference Proceedings 28th Annual Conference on Learning Theory

2015

Cited alongside, same era.

C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning requires rethinking generalization,” arXiv preprint arXiv:1611.03530

Original

2016

Cited alongside, same era.

T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao, “Why and when can deep–but not shallow–networks avoid the curse of dimensionality: a review,” arXiv preprint arXiv:1611.00740

Original

2016

Cited alongside, same era.

H. Mhaskar, Q. Liao, and T. Poggio, “Learning real and boolean functions: When is deep better than shallow?,” Center for Brains, Minds and Machines (CBMM) Memo No. 45, also in arXiv

2016

Later among the works it cites.

D. Soudry and Y. Carmon, “No bad local minima: Data independent training error guarantees for multilayer neural networks,” arXiv preprint arXiv:15605.08361

2016

Later among the works it cites.

K. Kawaguchi, “Deep learning without poor local minima,” in Advances in Neural Information Processing Systems (NIPS)

2016

Later among the works it cites.

C. Zhang, Q. Liao, A. Rakhlin, K. Sridharan, B. Miranda, N. Golowich, and T. Poggio, “Theory of deep learning iii: Generalization properties of sgd,” tech. rep., Center for Brains, Minds and Machines (CBMM), 2017

2017

Closest in time.