Representing smooth functions as compositions of near-identity functions with implications for deep network optimization

Approximation Capabilities of Multilayer Feedforward Networks

K. Hornik · 1991

Earlier work this paper cites.

For neural networks, function determines form

Francesca Albertini and Eduardo D Sontag · 1992

Earlier work this paper cites.

Reconstructing a neural net from its output

Charles Fefferman · 1994

Earlier work this paper cites.

Efficient agnostic learning of neural networks with bounded fan-in

Wee Sun Lee, Peter L Bartlett, and Robert C Williamson · 1996

Earlier work this paper cites.

Convex neural networks

Yoshua Bengio, Nicolas L Roux, Pascal Vincent, Olivier Delalleau, and Patrice Marcotte · 2006

Earlier work this paper cites.

Predicting parameters in deep learning

Misha Denil, Babak Shakibi, Laurent Dinh, Nando de Freitas, et al · 2013

Earlier work this paper cites.

Breaking the curse of dimensionality with convex neural networks

Original

Francis Bach · 2014

Cited alongside, same era.

On the number of linear regions of deep neural networks

Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio · 2014

Cited alongside, same era.

Gradient-based hyperparameter optimization through reversible learning

Dougal Maclaurin, David Duvenaud, and Ryan Adams · 2015

Cited alongside, same era.

Representation benefits of deep feedforward networks

Original

Matus Telgarsky · 2015

Cited alongside, same era.

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016

Cited alongside, same era.

Learning functions: when is deep better than shallow

Exponential expressivity in deep neural networks through transient chaos

Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli · 2016

Later among the works it cites.

Density estimation using real nvp

Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio · 2017

Later among the works it cites.

The reversible residual network: Backpropagation without storing activations

Aidan N. Gomez, Mengye Ren, Raquel Urtasun, and Roger B. Grosse · 2017

Later among the works it cites.

Identity matters in deep learning

M. Hardt and T. Ma · 2017

Later among the works it cites.

Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks

Original

Peter L Bartlett, David P Helmbold, and Philip M Long · 2018

Closest in time.