Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks

Approximation by superpositions of a sigmoidal function

Cybenko, George · 1989

Earlier work this paper cites.

Approximation capabilities of multilayer feedforward networks

Hornik, Kurt · 1991

Earlier work this paper cites.

Shallow vs. deep sum-product networks

Delalleau, O. and Bengio, Y · 2011

Earlier work this paper cites.

On the complexity of shallow and deep neural network classifiers

Bianchini, M. and Scarselli, F · 2014

Earlier work this paper cites.

On the expressive efficiency of sum product networks

Original

Martens, J. and Medabalimi, V · 2014

Earlier work this paper cites.

Deep residual learning for image recognition

Original

He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Jian · 2015

Cited alongside, same era.

On the expressive power of deep learning: A tensor analysis

Cohen, Nadav, Sharir, Or, and Shashua, Amnon · 2016

Cited alongside, same era.

The power of depth for feedforward neural networks

Eldan, Ronen and Shamir, Ohad · 2016

Cited alongside, same era.

Why deep neural networks?

Original

Liang, Shiyu and Srikant, R · 2016

Cited alongside, same era.

Why and when can deep–but not shallow–networks avoid the curse of dimensionality: a review

Original

Poggio, Tomaso, Mhaskar, Hrushikesh, Rosasco, Lorenzo, Miranda, Brando, and Liao, Qianli · 2016

Cited alongside, same era.

Exponential expressivity in deep neural networks through transient chaos

Poole, Ben, Lahiri, Subhaneil, Raghu, Maithreyi, Sohl-Dickstein, Jascha, and Ganguli, Surya · 2016

Closest in time.

Provable approximation properties for deep neural networks

Shaham, Uri, Cloninger, Alexander, and Coifman, Ronald R · 2016

Closest in time.

Benefits of depth in neural networks

Original

Telgarsky, Matus · 2016

Closest in time.

Error bounds for approximations with deep relu networks

Original

Yarotsky, Dmitry · 2016

Closest in time.