Fetching the paper…
Reading the bibliography…
The classical Universal Approximation Theorem holds for neural networks of arbitrary width and bounded depth.
Approximation by superpositions of a sigmoidal function
G. Cybenko · 1989
Earlier work this paper cites.
Approximation capabilities of multilayer feedforward networks
K. Hornik · 1991
Earlier work this paper cites.
Approximation theory of the MLP model in neural networks
A. Pinkus · 1999
Earlier work this paper cites.
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity
C. Yun, S. Sra, and A Jadbabaie · 1999
Earlier work this paper cites.
Networks of width one are universal classifiers
R. Rojas · 2003
Earlier work this paper cites.
Deep, Narrow Sigmoid Belief Networks Are Universal Approximators
I. Sutskever and G. E. Hinton · 2008
Earlier work this paper cites.
Deep Belief Networks are Compact Universal Approximators
N. Le Roux and Y. Bengio · 2010
Earlier work this paper cites.
Convolutional Deep Belief Networks on CIFAR-10
A. Krizhevsky · 2012
Earlier work this paper cites.
Deep, super-narrow neural network is a universal classifier
L. Szymanski and B. McCane · 2012
Earlier work this paper cites.
Rectifier Nonlinearities Improve Neural Network Acoustic Models
A. Maas, A. Hannun, and A. Ng · 2013
Earlier work this paper cites.
Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units
G. F. Montúfar · 2014
Cited alongside, same era.
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
K. He, X. Zhang, S. Ren, and J. Sun · 2015
Cited alongside, same era.
Empirical Evaluation of Rectified Activations in Convolutional Network
B. Xu, N. Wang, T. Chen, and M. Li · 2015
Cited alongside, same era.
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
D.-A. Clevert, T. Unterthiner, and S. Hochreiter · 2016
Cited alongside, same era.
Approximating Continuous Functions by ReLU Nets of Minimal Width
B. Hanin and M. Sellke · 2017
Cited alongside, same era.
ResNet with one-neuron hidden layers is a Universal Approximator
H. Lin and S. Jegelka · 2018
Later among the works it cites.
Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions
Q. Nguyen, M. C. Mukkamala, and M. Hein · 2018
Later among the works it cites.
Optimal approximation of piecewise smooth functions using deep ReLU neural networks
P. Petersen and F. Voigtlaender · 2018
Later among the works it cites.
Nonlinear Approximation and (Deep) ReLU Networks
I. Daubechies, R. DeVore, S. Foucart, B. Hanin, and G. Petrova · 2019
Closest in time.
How degenerate is the parametrization of neural networks with the ReLU activation function?
D. M. Elbrächter, J. Berner, and P. Grohs · 2019
Closest in time.
Deep, Skinny Neural Networks are not Universal Approximators
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Self-normalizing neural networks
G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter · 2017
Cited alongside, same era.
The Expressive Power of Neural Networks: A View from the Width
Z. Lu, H. Pu, F. Wang, Z. Hu, and L. Wang · 2017
Cited alongside, same era.
Searching for Activation Functions
P. Ramachandran, B. Zoph, and Q. V. Le · 2017
Cited alongside, same era.
Understanding Deep Neural Networks with Rectified Linear Units
R. Arora, A. Basu, P. Mianjy, and A. Mukherjee · 2018
Cited alongside, same era.
On decision regions of narrow deep neural networks
H.-P. Beise, S. D. Da Cruz, and U. Schröder · 2018
Cited alongside, same era.
Efficient Approximation of Deep ReLU Networks for Functions on Low Dimensional Manifolds
M. Chen, H. Jiang, W. Liao, and T. Zhao
Cited in the paper.
Residual Flows for Invertible Generative Modeling
R. T. Q. Chen, J. Behrmann, D. K. Duvenaud, and J.-H. Jacobsen
Cited in the paper.
J. Johnson · 2019
Closest in time.
Gradient Dynamics of Shallow Univariate ReLU Networks
F. Williams, M. Trager, D. Panozzo, C. Silva, D. Zorin, and J. Bruna · 2019
Closest in time.
N. Boullé, Y. Nakatsukasa, and A. Townsend · 2020
Closest in time.
Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks
A. Molina, P. Schramowski, and K. Kersting · 2020
Closest in time.