Fetching the paper…
Reading the bibliography…
This paper explores the expressive power of deep neural networks for a diverse range of activation functions.
Approximation by superpositions of a sigmoidal function
George Cybenko · 1989
Earlier work this paper cites.
Multilayer feedforward networks are universal approximators
Kurt Hornik, Maxwell Stinchcombe, and Halbert White · 1989
Earlier work this paper cites.
Approximation capabilities of multilayer feedforward networks
Kurt Hornik · 1991
Earlier work this paper cites.
Universal approximation bounds for superpositions of a sigmoidal function
Andrew R. Barron · 1993
Earlier work this paper cites.
Ion Victor Gosea and Athanasios C. Antoulas · 2005
Earlier work this paper cites.
Rectified linear units improve restricted Boltzmann machines
Vinod Nair and Geoffrey E. Hinton · 2010
Earlier work this paper cites.
Deep sparse rectifier neural networks
Xavier Glorot, Antoine Bordes, and Yoshua Bengio · 2011
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Rectifier nonlinearities improve neural network acoustic models
Andrew L Maas, Awni Y Hannun, and Andrew Y Ng · 2013
Earlier work this paper cites.
Fast and accurate deep network learning by exponential linear units (ELUs)
Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter · 2016
Earlier work this paper cites.
Gaussian error linear units (GELUs)
Dan Hendrycks and Kevin Gimpel · 2016
Earlier work this paper cites.
Continuously differentiable exponential linear units
Jonathan T. Barron · 2017
Earlier work this paper cites.
Sigmoid-weighted linear units for neural network function approximation in reinforcement learning
Stefan Elfwing, Eiji Uchibe, and Kenji Doya · 2017
Earlier work this paper cites.
Self-normalizing neural networks
Günter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter · 2017
Earlier work this paper cites.
Searching for activation functions
Prajit Ramachandran, Barret Zoph, and Quoc V. Le · 2017
Earlier work this paper cites.
Error bounds for approximations with deep ReLU networks
Dmitry Yarotsky · 2017
Cited alongside, same era.
Approximation and estimation for high-dimensional deep learning networks
Andrew R. Barron and Jason M. Klusowski · 2018
Cited alongside, same era.
Construction of neural networks for realization of localized deep learning
Charles K. Chui, Shao-Bo Lin, and Ding-Xuan Zhou · 2018
Cited alongside, same era.
Optimal approximation of continuous functions by very deep ReLU networks
Dmitry Yarotsky · 2018
Cited alongside, same era.
Optimal approximation with sparsely connected deep neural networks
Helmut Bölcskei, Philipp Grohs, Gitta Kutyniok, and Philipp Petersen · 2019
Cited alongside, same era.
Efficient approximation of deep ReLU networks for functions on low dimensional manifolds
Soft-Root-Sign: A new bounded neural activation function
Dandan Li and Yuan Zhou · 2020
Later among the works it cites.
Mish: A self regularized non-monotonic activation function
Diganta Misra · 2020
Later among the works it cites.
Adaptive approximation and generalization of deep neural network with intrinsic dimensionality
Ryumei Nakada and Masaaki Imaizumi · 2020
Later among the works it cites.
Deep network approximation characterized by number of neurons
Zuowei Shen, Haizhao Yang, and Shijun Zhang · 2020
Later among the works it cites.
Deep neural network approximation via function compositions
Shijun Zhang · 2020
Later among the works it cites.
Deep network approximation for smooth functions
Jianfeng Lu, Zuowei Shen, Haizhao Yang, and Shijun Zhang · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Minshuo Chen, Haoming Jiang, Wenjing Liao, and Tuo Zhao · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
Hadrien Montanelli and Haizhao Yang · 2019
Cited alongside, same era.
Nonlinear approximation via compositions
Zuowei Shen, Haizhao Yang, and Shijun Zhang · 2019
Cited alongside, same era.
Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
Taiji Suzuki · 2019
Cited alongside, same era.
XLNet: Generalized autoregressive pretraining for language understanding
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le · 2019
Cited alongside, same era.
Universality of deep convolutional neural networks
Ding-Xuan Zhou · 2019
Cited alongside, same era.
Optimal approximation rate of ReLU networks in terms of width and depth
Zuowei Shen, Haizhao Yang, and Shijun Zhang · 2021
Later among the works it cites.
High-order approximation rates for shallow neural networks with cosine and ReLU k
Jonathan W. Siegel and Jinchao Xu · 2021
Later among the works it cites.
Approximation analysis of convolutional neural networks
Chenglong Bao, Qianxiao Li, Zuowei Shen, Cheng Tai, Lei Wu, and Xueshuang Xiang · 2022
Later among the works it cites.
Improved bounds on neural complexity for representing piecewise linear functions
Kuan-Lin Chen, Harinath Garudadri, and Bhaskar D Rao · 2022
Later among the works it cites.
Approximation spaces of deep neural networks
Rémi Gribonval, Gitta Kutyniok, Morten Nielsen, and Felix Voigtlaender · 2022
Later among the works it cites.
Neural network architecture beyond width and depth
Zuowei Shen, Haizhao Yang, and Shijun Zhang · 2022
Later among the works it cites.
Deep learning via dynamical systems: An approximation perspective
Qianxiao Li, Ting Lin, and Zuowei Shen · 2023
Closest in time.
Quadratic features and deep architectures for chunking
Joseph Turian, James Bergstra, and Yoshua Bengio · 2062
Closest in time.