2019

Estimation of a function of low local dimensionality by deep neural networks

Kohler, Michael, Krzyzak, Adam, Langer, Sophie

Understand

Deep neural networks (DNNs) achieve impressive results for complicated tasks like object detection on images and speech recognition.

Motivated by this practical success, there is now a strong interest in showing good theoretical properties of DNNs.
To describe for which tasks DNNs perform well and when they fail, it is a key challenge to understand their performance.
The aim of this paper is to contribute to the current statistical theory of DNNs.

Built on

Optimal nonparametric inference via deep neural network
Original
Liu, R., Boukai, B., and Shang, Z. (2019) · 1902
Earlier work this paper cites.
Adaptive approximation and estimation of deep neural network to intrinsic dimensionality
Original
Nakada, R. and Imaizumi, M. (2019) · 1907
Earlier work this paper cites.
On the rate of convergence of fully connected deep neural network regression estimates
Original
Kohler, M. and Langer, S. (2020) · 1908
Earlier work this paper cites.
Deep relu network approximation of functions on a manifold
Original
Schmidt-Hieber, J. (2019) · 1908
Earlier work this paper cites.
Distribution-free consistency results in nonparametric discrimination and regression function estimation
Devroye, L. P. and Wagner, T. J. (1980) · 1980
Earlier work this paper cites.
Optimal global rates of convergence for nonparametric regression
Stone, C. J. (1982) · 1982
Earlier work this paper cites.
Multivariate adaptive regression splines
Friedman, J. H. (1991) · 1991
Earlier work this paper cites.
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., and Lugosi, G. (1996) · 1996
Earlier work this paper cites.
The “independent components” of natural scenes are edge filters
Bell, A. J. and Sejnowski, T. J. (1997) · 1997
Earlier work this paper cites.
Local dimensionality reduction for locally weighted learning
Schaal, S. and Vijayakumar, S. (1997) · 1997
Earlier work this paper cites.
Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results
Scarselli, F. and Tsoi, A. C. (1998) · 1998
Earlier work this paper cites.

Similar

Information-theoretic determination of minimax rates of convergence
Yang, Y. and Barron, A. (1999) · 1999
Cited alongside, same era.
A global geometric framework for nonlinear dimensionality reduction
Tenenbaum, J. B., Silva, V., and Langford, J. C. (2000) · 2000
Cited alongside, same era.
A Distribution-Free Theory of Nonparametric Regression
Györfi, L., Kohler, M., Krzyżak, A., and Walk, H. (2002) · 2002
Cited alongside, same era.
Radial basis functions for the multivariate interpolation of large scattered data sets
Lazzaro, D. and Montefusco, L. B. (2002) · 2002
Cited alongside, same era.
Laplacian eigenmaps for dimensionality reduction and data representation
Belkin, M. and Niyogi, P. (2003) · 2003
Cited alongside, same era.
Local polynomial regression on unknown manifolds
Bickel, P. J. and Li, B. (2007) · 2007
Cited alongside, same era.
Estimation of a regression function by maxima of minima of linear functions
Bagirov, A. M., Clausen, C., and Kohler, M. (2009) · 2009
Cited alongside, same era.
Local dimensionality reduction for non-parametric regression
Hoffmann, H., Schaal, S., and Vijayakumar, S. (2009) · 2009
Cited alongside, same era.
k-nn regression adapts to local intrinsic dimension
Kpotufe, S. (2011) · 2011
Cited alongside, same era.
Event labeling combining ensemble detectors and background knowledge
Fanaee-T, H. and Gama, J. (2013) · 2013
Cited alongside, same era.

Then

Adaptivity to local smoothness and dimension in kernel regression
Kpotufe, S. and Garg, V. (2013) · 2013
Later among the works it cites.
Optimal global rates of convergence for noiseless regression estimation problems with adaptively chosen design
Kohler, M. (2014) · 2014
Later among the works it cites.
Deep learning in neural networks: An overview
Schmidhuber, J. (2015) · 2015
Later among the works it cites.
Areslab: Adaptive regression splines toolbox for matlab/octave
Jekabsons, G. (2016) · 2016
Later among the works it cites.
Nonparametric estimation of a function from noiseless observations at random points
Bauer, B., Devroye, L., Kohler, M., Krzyżak, A., and Walk, H. (2017) · 2017
Later among the works it cites.
Nonparametric regression based on hierarchical interaction models
Kohler, M. and Krzyżak, A. (2017) · 2017
Later among the works it cites.
On deep learning as a remedy for the curse of dimensionality in nonparametric regression
Bauer, B. and Kohler, M. (2019) · 2019
Closest in time.
A comparison of deep networks with relu activation function and linear spline-type methods
Eckle, K. and Schmidt-Hieber, J. (2019) · 2019
Closest in time.
Deep neural networks learn non-smooth functions effectively
Imaizumi, M. and Fukumizu, K. (2019) · 2019
Closest in time.
Nonparametric regression using deep neural networks with relu activation function
Schmidt-Hieber, J. (2020) · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…