Fetching the paper…
Reading the bibliography…
We establish the fundamental limits in the approximation of Lipschitz functions by deep ReLU neural networks with finite-precision weights.
S. Tewksbury and R. Hallock, “Oversampled, linear predictive and noise-shaping coders of order N > 1 N>1 ,” IEEE Transactions on Circuits and Systems , vol. 25, no. 7, pp. 436–447, 1978
1978
Earlier work this paper cites.
H. T. Siegelmann and E. D. Sontag, “On the computational power of neural nets,” in Proceedings of the Fifth Annual Workshop on Computational Learning Theory , 1992, pp. 440–449
1992
Earlier work this paper cites.
P. L. Bartlett, V. Maiorov, and R. Meir, “Almost linear VC-dimension bounds for piecewise polynomial networks,” Neural Computation , vol. 10, no. 8, pp. 2159–2173, 1998
1998
Earlier work this paper cites.
M. Telgarsky, “Benefits of depth in neural networks,” in 29th Annual Conference on Learning Theory , ser. Proceedings of Machine Learning Research, vol. 49, 23–26 Jun 2016, pp. 1517–1539
2016
Earlier work this paper cites.
D. Yarotsky, “Error bounds for approximations with deep ReLU networks,” Neural Networks , vol. 94, pp. 103 – 114, 2017
2017
Earlier work this paper cites.
P. Petersen and F. Voigtlaender, “Optimal approximation of piecewise smooth functions using deep ReLU neural networks,” Neural Networks , vol. 108, pp. 296–330, 2018
2018
Earlier work this paper cites.
——, “Optimal approximation of continuous functions by very deep ReLU networks,” in Proceedings of the 31st Conference On Learning Theory , ser. Proceedings of Machine Learning Research, vol. 75, 06–09 Jul 2018, pp. 639–649
2018
Earlier work this paper cites.
2019
Earlier work this paper cites.
P. L. Bartlett, N. Harvey, C. Liaw, and A. Mehrabian, “Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks,” Journal of Machine Learning Research , vol. 20, no. 63, pp. 1–17, 2019
2019
Cited alongside, same era.
M. J. Wainwright, High-dimensional statistics: A non-asymptotic viewpoint , 2nd ed. Cambridge, UK: Cambridge University Press, 2019, vol. 48
2019
Cited alongside, same era.
——, “Nonlinear approximation via compositions,” Neural Networks , vol. 119, pp. 74–84, 2019
2019
Cited alongside, same era.
J. Schmidt-Hieber, “Nonparametric regression using deep neural networks with ReLU activation function,” The Annals of Statistics , vol. 48, no. 4, pp. 1875 – 1897, 2020
2020
Cited alongside, same era.
R. Nakada and M. Imaizumi, “Adaptive approximation and generalization of deep neural network with intrinsic dimensionality,” Journal of Machine Learning Research , vol. 21, no. 174, pp. 1–38, 2020
M. Kohler and S. Langer, “On the rate of convergence of fully connected deep neural network regression estimates,” The Annals of Statistics , vol. 49, no. 4, pp. 2231 – 2249, 2021
2021
Later among the works it cites.
D. Elbrächter, D. Perekrestenko, P. Grohs, and H. Bölcskei, “Deep neural network approximation theory,” IEEE Transactions on Information Theory , vol. 67, no. 5, pp. 2581–2623, May 2021
2021
Later among the works it cites.
J. Lu, Z. Shen, H. Yang, and S. Zhang, “Deep network approximation for smooth functions,” SIAM Journal on Mathematical Analysis , vol. 53, no. 5, pp. 5465–5506, 2021
2021
Later among the works it cites.
M. Chen, H. Jiang, W. Liao, and T. Zhao, “Nonparametric regression on low-dimensional manifolds using deep ReLU networks: Function approximation and statistical recovery,” Information and Inference: A Journal of the IMA , vol. 11, no. 4, pp. 1203–1253, 03 2022
2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2020
Cited alongside, same era.
D. Yarotsky and A. Zhevnerchuk, “The phase diagram of approximation rates for deep neural networks,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 13 005–13 015
2020
Cited alongside, same era.
Z. Shen, H. Yang, and S. Zhang, “Deep network approximation characterized by number of neurons,” Communications in Computational Physics , vol. 28, no. 5, pp. 1768–1811, 2020
2020
Cited alongside, same era.
I. Gühring and M. Raslan, “Approximation rates for neural networks with encodable weights in smoothness spaces,” Neural Networks , vol. 134, pp. 107–130, 2021
2021
Cited alongside, same era.
Z. Shen, H. Yang, and S. Zhang, “Optimal approximation rate of ReLU networks in terms of width and depth,” Journal de Mathématiques Pures et Appliquées , vol. 157, pp. 101–135, 2022
2022
Later among the works it cites.
G. Vardi, G. Yehudai, and O. Shamir, “Width is less important than depth in ReLU neural networks,” in Proceedings of Thirty Fifth Conference on Learning Theory , ser. Proceedings of Machine Learning Research, vol. 178. PMLR, 02–05 Jul 2022, pp. 1249–1281
2022
Later among the works it cites.
I. Daubechies, R. DeVore, S. Foucart, B. Hanin, and G. Petrova, “Nonlinear approximation and (deep) ReLU networks,” Constructive Approximation , vol. 55, no. 1, pp. 127–172, 2022
2022
Later among the works it cites.