Fetching the paper…
Reading the bibliography…
This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning.
I. M. Gel’fand, G. E. Shilov, Generalized Functions, Vol. 1: Properties and Operations, Academic Press, New York, 1964
1964
Earlier work this paper cites.
L. Schwartz, Théorie des Distributions, nouvelle Edition, Hermann, Paris, 1966
1966
Earlier work this paper cites.
F. Trèves, Tological Vector Spaces, Distributions and Kernels, Academic Press, 1967
1967
Earlier work this paper cites.
1970
Earlier work this paper cites.
doi:10.1109/TNS.1974.6499235
L. A. Shepp, B. F. Logan, The Fourier reconstruction of a head section, Nucl. Sci. IEEE Trans. 21 (3) (1974) 21–43 · 1974
Earlier work this paper cites.
doi:10.1007/BF01252856. URL http://dx.doi.org/10.1007/BF01252856
A. Hertle, Continuity of the radon transform and its inverse on Euclidean space, Math. Zeitschrift 184 (2) (1983) 165–192 · 1983
Earlier work this paper cites.
doi:10.1109/ICNN.1988.23901
B. Irie, S. Miyake, Capabilities of three-layered perceptrons, in: IEEE Int. Conf. Neural Networks, IEEE, 1988, pp. 641–648 · 1988
Earlier work this paper cites.
doi:10.1109/IJCNN.1989.118639
S. M. Carroll, B. W. Dickinson, Construction of neural nets using the Radon transform, in: Int. Jt. Conf. Neural Networks, 1989. IJCNN., Vol. 1, IEEE, 1989, pp. 607–611 · 1989
Earlier work this paper cites.
doi:10.1016/0893-6080(89)90003-8. URL http://www.sciencedirect.com/science/article/pii/0893608089900038
K.-I. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2 (3) (1989) 183–192 · 1989
Earlier work this paper cites.
doi:10.1016/0893-6080(91)90075-G. URL http://www.sciencedirect.com/science/article/pii/089360809190075G
Y. Ito, Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory, Neural Networks 4 (3) (1991) 385–394 · 1991
Earlier work this paper cites.
W. Rudin, Functional Analysis, 2nd Edition, Higher Mathematics Series, McGraw-Hill Education, 1991
1991
Earlier work this paper cites.
doi:10.1016/0196-8858(92)90016-P. URL http://www.sciencedirect.com/science/article/pii/019688589290016P
H. Mhaskar, C. A. Micchelli, Approximation by superposition of sigmoidal and radial basis functions, Adv. Appl. Math. 13 (3) (1992) 350–373 · 1992
Earlier work this paper cites.
doi:10.1214/aos/1176348546
L. K. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training, Ann. Stat. 20 (1) (1992) 608–613 · 1992
Earlier work this paper cites.
doi:10.1016/S0893-6080(05)80131-5. URL http://www.sciencedirect.com/science/article/pii/S0893608005801315
M. Leshno, V. Y. Lin, A. Pinkus, S. Schocken, Multilayer feedforward networks with a nonpolynomial activation function can approximate any function, Neural Networks 6 (6) (1993) 861–867 · 1993
Earlier work this paper cites.
doi:10.1109/18.256500
A. R. Barron, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inf. Theory 39 (3) (1993) 930–945 · 1993
Earlier work this paper cites.
doi:10.1007/978-3-642-61859-8
K. Yosida, Functional Analysis, 6th Edition, Springer-Verlag Berlin Heidelberg, 1995 · 1995
Cited alongside, same era.
M. Holschneider, Wavelets: An Analysis Tool, Oxford mathematical monographs, The Clarendon Press, 1995
1995
Cited alongside, same era.
doi:10.1016/0893-6080(96)00000-7. URL http://www.sciencedirect.com/science/article/pii/0893608096000007
N. Murata, An Integral representation of functions using three-layered betworks and their approximation bounds, Neural Networks 9 (6) (1996) 947–956 · 1996
Cited alongside, same era.
doi:10.1006/acha.1998.0248. URL http://www.sciencedirect.com/science/article/pii/S1063520398902482
E. J. Candès, Harmonic analysis of neural networks, Appl. Comput. Harmon. Anal. 6 (2) (1999) 197–218 · 1998
Cited alongside, same era.
E. J. Candès, Ridgelets: theory and applications, Ph.D. thesis, Standford University (1998)
1998
Cited alongside, same era.
doi:10.1007/978-3-642-14606-0
W. Yuan, W. Sickel, D. Yang, Morrey and Campanato Meet Besov, Lizorkin and Triebel, Lecture Notes in Mathematics, Springer Berlin Heidelberg, 2010 · 2010
Later among the works it cites.
X. Glorot, A. Bordes, Y. Bengio, Deep sparse rectifier neural networks, in: 14th Int. Conf. Artif. Intell. Stat. (AISTATS 2011), Vol. 15, JMLR W&CP, Fort Lauderdale, FL, USA, 2011, pp. 315–323. URL http://jmlr.org/proceedings/papers/v15/glorot11a/glorot11a.pdf
2011
Later among the works it cites.
doi:10.1007/978-1-4419-6055-9
S. Helgason, Integral Geometry and Radon Transforms, Springer-Verlag New York, 2011 · 2011
Later among the works it cites.
doi:10.1007/978-0-387-70914-7
H. Brezis, Functional Analysis, Sobolev Spaces and Partial Differential Equations, 1st Edition, Universitext, Springer-Verlag New York, 2011 · 2011
Later among the works it cites.
A. Krizhevsky, I. Sutskever, G. E. Hinton, ImageNet classification with deep convolutional neural networks, in: F. Pereira, C. J. C. Burges, L. Bottou, K. Q. Weinberger (Eds.), Adv. Neural Inf. Process. Syst. 25, Curran Associates, Inc., 2012, pp. 1097–1105. URL http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
doi:10.1007/BF02475988. URL http://dx.doi.org/10.1007/BF02475988
B. Rubin, The Calderón reproducing formula, windowed X-ray transforms, and radon transforms in L p L^{p} -spaces, J. Fourier Anal. Appl. 4 (2) (1998) 175–197 · 1998
Cited alongside, same era.
doi:10.1017/S0962492900002919
A. Pinkus, Approximation theory of the MLP model in neural networks, Acta Numer. 8 (1999) 143–195 · 1999
Cited alongside, same era.
doi:10.1073/pnas.96.5.1828
D. L. Donoho, Tight frames of k k -plane ridgelets and the problem of representing objects that are smooth away from d d -dimensional singularities in ℝ n \mathbb{R}^{n} , Proc. Natl. Acad. Sci. United States Am. 96 (5) (1999) 1828–1833 · 1999
Cited alongside, same era.
doi:10.1006/jath.2001.3568. URL http://www.sciencedirect.com/science/article/pii/S0021904501935683
D. L. Donoho, Ridge functions and orthonormal ridgelets, J. Approx. Theory 111 (2) (2001) 143–179 · 2001
Cited alongside, same era.
doi:10.1016/j.acha.2004.03.003. URL http://www.sciencedirect.com/science/article/pii/S1063520304000168
B. Rubin, Convolution–backprojection method for the k k -plane transform, and Calderón’s identity for ridgelet transforms, Appl. Comput. Harmon. Anal. 16 (3) (2004) 231–242 · 2004
Cited alongside, same era.
doi:10.1016/j.jat.2006.12.009. URL http://linkinghub.elsevier.com/retrieve/pii/S0021904507000081
P. C. Kainen, V. Kůrková, A. Vogt, A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves, J. Approx. Theory 147 (1) (2007) 1–10 · 2006
Cited alongside, same era.
doi:10.1007/978-0-387-09432-8
L. Grafakos, Classical Fourier Analysis, 2nd Edition, Graduate Texts in Mathematics, Springer New York, 2008 · 2008
Cited alongside, same era.
2012
Later among the works it cites.
doi:10.1016/j.neunet.2012.05.002. URL http://www.sciencedirect.com/science/article/pii/S0893608012001311
V. Kůrková, Complexity estimates based on integral transforms induced by computational units, Neural Netw. 33 (2012) 160–7 · 2012
Later among the works it cites.
I. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, Y. Bengio, Maxout networks, in: 30th Int. Conf. Mach. Learn., Vol. 28, JMLR W&CP, 2013, pp. 1319–1327. URL http://jmlr.csail.mit.edu/proceedings/papers/v28/goodfellow13.pdf
2013
Later among the works it cites.
doi:10.1109/ICASSP.2013.6639346
G. E. Dahl, T. N. Sainath, G. E. Hinton, Improving deep neural networks for LVCSR using rectified linear units and dropout, in: Acoust. Speech Signal Process. (ICASSP), 2013 IEEE Int. Conf., IEEE, 2013, pp. 8609–8613 · 2013
Later among the works it cites.
A. L. Maas, A. Y. Hannun, A. Y. Ng, Rectifier nonlinearities improve neural network acoustic models, in: ICML 2013 Work. Deep Learn. Audio, Speech, Lang. Process., Atlanta, 2013. URL https://sites.google.com/site/deeplearningicml2013/relu_hybrid_icml2013_final.pdf
2013
Later among the works it cites.
doi:10.1109/ICASSP.2013.6638312
M. D. Zeiler, M. Ranzato, R. Monga, M. Z. Mao, K. Yang, Q. Viet Le, P. Nguyen, A. W. Senior, V. Vanhoucke, J. Dean, G. E. Hinton, On rectified linear units for speech processing, in: Acoust. Speech Signal Process. (ICASSP), 2013 IEEE Int. Conf., IEEE, Vancouver, BC, 2013, pp. 3517–3521 · 2013
Later among the works it cites.
doi:10.1007/978-3-642-36657-4
P. C. Kainen, V. Kůrková, M. Sanguineti, Approximating multivariable functions by feedforward neural nets, in: M. Bianchini, M. Maggini, L. C. Jain (Eds.), Handb. Neural Inf. Process., Vol. 49 of Intelligent Systems Reference Library, Springer Berlin Heidelberg, 2013, pp. 143–181 · 2013
Later among the works it cites.
doi:10.1080/10652469.2013.853057
S. Kostadinova, S. Pilipović, K. Saneva, J. Vindas, The ridgelet transform of distributions, Integr. Transform. Spec. Funct. 25 (5) (2014) 344–358 · 2013
Later among the works it cites.
doi:10.1007/978-3-319-11179-7_68
S. Sonoda, N. Murata, Sampling hidden parameters from oracle distribution, in: 24th Int. Conf. Artif. Neural Networks, Vol. 8681, Springer International Publishing, Hamburg, Germany, 2014, pp. 539–546 · 2014
Later among the works it cites.
doi:10.1007/978-3-319-14618-8_13
S. Kostadinova, S. Pilipović, K. Saneva, J. Vindas, The Ridgelet Transform and Quasiasymptotic Behavior of Distributions, Oper. Theory Adv. Appl. 245 (2015) 185–197 · 2015
Closest in time.