Fetching the paper…
Reading the bibliography…
In this paper, we investigate the expressivity and approximation properties of deep neural networks employing the ReLU$^k$ activation function for $k \geq 2$.
On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition
A. N. Kolmogorov · 1957
Earlier work this paper cites.
On inverses of vandermonde and confluent vandermonde matrices iii
W. Gautschi · 1978
Earlier work this paper cites.
Approximation by superpositions of a sigmoidal function
G. Cybenko · 1989
Earlier work this paper cites.
Multilayer feedforward networks are universal approximators
K. Hornik, M. Stinchcombe, and H. White · 1989
Earlier work this paper cites.
Approximation by ridge functions and neural networks with one hidden layer
C. K. Chui and X. Li · 1992
Earlier work this paper cites.
Universal approximation bounds for superpositions of a sigmoidal function
A. R. Barron · 1993
Earlier work this paper cites.
Constructive approximation
R. A. DeVore and G. G. Lorentz · 1993
Earlier work this paper cites.
Multilayer feedforward networks with a nonpolynomial activation function can approximate any function
M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken · 1993
Earlier work this paper cites.
Approximation properties of a multilayered feedforward artificial neural network
H. N. Mhaskar · 1993
Earlier work this paper cites.
Degree of approximation by neural and translation networks with a single hidden layer
H. N. Mhaskar and C. A. Micchelli · 1995
Earlier work this paper cites.
Limitations of the approximation capabilities of neural networks with one hidden layer
C. K. Chui, X. Li, and H. N. Mhaskar · 1996
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Learning theory: an approximation theory viewpoint
F. Cucker and D. X. Zhou · 2007
Earlier work this paper cites.
Natural language processing (almost) from scratch
R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa · 2011
Earlier work this paper cites.
Shallow vs. deep sum-product networks
O. Delalleau and Y. Bengio · 2011
Earlier work this paper cites.
Strategies for training large scale neural network language models
T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Černockỳ · 2011
Earlier work this paper cites.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Improvements to deep convolutional neural networks for lvcsr
T. N. Sainath, B. Kingsbury, A.-r. Mohamed, G. E. Dahl, G. Saon, H. Soltau, T. Beran, A. Y. Aravkin, and B. Ramabhadran · 2013
Earlier work this paper cites.
Question answering with subgraph embeddings
A. Bordes, S. Chopra, and J. Weston · 2014
Earlier work this paper cites.
On using very large target vocabulary for neural machine translation
S. Jean, K. Cho, R. Memisevic, and Y. Bengio · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
I. Sutskever, O. Vinyals, and Q. V. Le · 2014
Cited alongside, same era.
Representation benefits of deep feedforward networks
M. Telgarsky · 2015
Cited alongside, same era.
The power of depth for feedforward neural networks
R. Eldan and O. Shamir · 2016
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Depth separation in ReLU networks for approximating smooth non-linear functions
Universality of deep convolutional neural networks
D.-X. Zhou · 2020
Later among the works it cites.
Theory of deep convolutional neural networks iii: Approximating radial functions
T. Mao, Z. Shi, and D.-X. Zhou · 2021
Later among the works it cites.
Optimal approximation rates and metric entropy of ReLUk and cosine networks
J. W. Siegel and J. Xu · 2021
Later among the works it cites.
Power series expansion neural network
Q. Chen, W. Hao, and J. He · 2022
Later among the works it cites.
Approximation properties of deep relu cnns
J. He, L. Li, and J. Xu · 2022
Later among the works it cites.
Relu deep neural networks from the hierarchical basis perspective
J. He, L. Li, and J. Xu · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
I. Safran and O. Shamir · 2016
Cited alongside, same era.
The expressive power of neural networks: A view from the width
Z. Lu, H. Pu, F. Wang, Z. Hu, and L. Wang · 2017
Cited alongside, same era.
Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review
T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao · 2017
Cited alongside, same era.
Error bounds for approximations with deep ReLU networks
D. Yarotsky · 2017
Cited alongside, same era.
Understanding deep neural networks with rectified linear units
R. Arora, A. Basu, P. Mianjy, and A. Mukherjee · 2018
Cited alongside, same era.
Approximation by combinations of ReLU and squared ReLU ridge functions with ℓ 1 \ell^{1} and ℓ 0 \ell^{0} controls
J. M. Klusowski and A. R. Barron · 2018
Cited alongside, same era.
Deep neural networks for rotation-invariance approximation and learning
C. K. Chui, S.-B. Lin, and D.-X. Zhou · 2019
Cited alongside, same era.
Uniform approximation rates and metric entropy of shallow neural networks
L. Ma, J. W. Siegel, and J. Xu · 2022
Later among the works it cites.
Exponential relu dnn expression of holomorphic maps in high dimension
J. A. Opschoor, C. Schwab, and J. Zech · 2022
Later among the works it cites.
Optimal approximation rate of ReLU networks in terms of width and depth
Z. Shen, H. Yang, and S. Zhang · 2022
Later among the works it cites.
Optimal approximation rates for deep ReLU neural networks on sobolev spaces
J. W. Siegel · 2022
Later among the works it cites.
High-order approximation rates for shallow neural networks with cosine and ReLUk activation functions
J. W. Siegel and J. Xu · 2022
Later among the works it cites.
Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks
J. W. Siegel and J. Xu · 2022
Later among the works it cites.
Encoding of data sets and algorithms
K. Doctor, T. Mao, and H. Mhaskar · 2023
Closest in time.
J. He · 2023
Closest in time.
Deep neural networks and finite elements of any order on arbitrary dimensions
J. He and J. Xu · 2023
Closest in time.
Approximating functions with multi-features by deep convolutional neural networks
T. Mao, Z. Shi, and D.-X. Zhou · 2023
Closest in time.
Rates of approximation by ReLU shallow neural networks
T. Mao and D.-X. Zhou · 2023
Closest in time.
Approximation of nonlinear functionals using deep ReLU networks
L. Song, J. Fan, D.-R. Chen, and D.-X. Zhou · 2023
Closest in time.
Optimal rates of approximation by shallow ReLU k
Y. Yang and D.-X. Zhou · 2023
Closest in time.