Understand
We show that minimum-norm interpolation in the Reproducing Kernel Hilbert Space corresponding to the Laplace kernel is not consistent if input dimension is constant.
- The lower bound holds for any choice of kernel bandwidth, even if selected based on data.
- The result supports the empirical observation that minimum-norm interpolation (that is, exact fit to training data) in RKHS generalizes well for some high-dimensional datasets, but not for low-dimensional ones.
Built on
Introduction to Fourier analysis on Euclidean spaces (PMS-32)
Elias M Stein and Guido Weiss · 1971
Earlier work this paper cites.
A distribution-free theory of nonparametric regression
László Györfi, Michael Kohler, Adam Krzyzak, and Harro Walk · 2006
Earlier work this paper cites.
The spectrum of kernel random matrices
Noureddine El Karoui · 2010
Earlier work this paper cites.
Similar
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 2016
Cited alongside, same era.
A First Course in Sobolev Spaces, Second Edition
Giovanni Leoni · 2017
Cited alongside, same era.
Overfitting or perfect fitting? risk bounds for classification and regression rules that interpolate
Mikhail Belkin, Daniel Hsu, and Partha Mitra
Cited in the paper.
To understand deep learning we need to understand kernel learning
Mikhail Belkin, Siyuan Ma, and Soumik Mandal
Cited in the paper.
Does data interpolation contradict statistical optimality?
Mikhail Belkin, Alexander Rakhlin, and Alexandre B Tsybakov
Cited in the paper.
Then
Explaining the success of adaboost and random forests as interpolating classifiers
Abraham J Wyner, Matthew Olson, Justin Bleich, and David Mease · 2017
Later among the works it cites.
Just interpolate: Kernel” ridgeless” regression can generalize
Tengyuan Liang and Alexander Rakhlin · 2018
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…