Learning in High Dimension Always Amounts to Extrapolation

On the inductive bias of neural tangent kernels

Original

Bietti, A. and Mairal, J. (2019) · 1905

Earlier work this paper cites.

Interpolation und extrapolation von stationaren zufalligen folgen

Kolmogoroff, A. (1941) · 1941

Earlier work this paper cites.

Extrapolation, interpolation, and smoothing of stationary time series, with engineering applications

Wiener, N. (1949) · 1949

Earlier work this paper cites.

Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis

Kruskal, J. B. (1964) · 1964

Earlier work this paper cites.

Representation and analysis of signals part xxi. the intrinsic dimensionality of signal collections

Bennett, R. S. (1965) · 1965

Earlier work this paper cites.

Extensions of lipschitz mappings into a hilbert space 26

Johnson, W. B. and Lindenstrauss, J. (1984) · 1984

Earlier work this paper cites.

On a conjecture of re miles about the convex hull of random points

Buchta, C. (1986) · 1986

Earlier work this paper cites.

On the shape of the convex hull of random points

Bárány, I. and Füredi, Z. (1988) · 1988

Earlier work this paper cites.

Probability thatn random points are in convex position

Valtr, P. (1995) · 1995

Earlier work this paper cites.

The probability that n random points in a triangle are in convex position

Valtr, P. (1996) · 1996

Cited alongside, same era.

Nonlinear approximation

DeVore, R. A. (1998) · 1998

Cited alongside, same era.

Nonlinear dimensionality reduction by locally linear embedding

Roweis, S. T. and Saul, L. K. (2000) · 2000

Cited alongside, same era.

The isomap algorithm and topological stability

Balasubramanian, M., Schwartz, E. L., Tenenbaum, J. B., de Silva, V., and Langford, J. C. (2002) · 2002

Cited alongside, same era.

Laplacian eigenmaps for dimensionality reduction and data representation

Belkin, M. and Niyogi, P. (2003) · 2003

Cited alongside, same era.

An elementary proof of a theorem of johnson and lindenstrauss

Dasgupta, S. and Gupta, A. (2003) · 2003

Cited alongside, same era.

Pattern recognition

Bishop, C. M. (2006) · 2006

Later among the works it cites.

Mlle: Modified locally linear embedding using multiple weights

Zhang, Z. and Wang, J. (2007) · 2007

Later among the works it cites.

Visualizing data using t-sne

Van der Maaten, L. and Hinton, G. (2008) · 2008

Later among the works it cites.

Random convex hulls and extreme value statistics

Majumdar, S. N., Comtet, A., and Randon-Furling, J. (2010) · 2010

Later among the works it cites.

To understand deep learning we need to understand kernel learning

Belkin, M., Ma, S., and Mandal, S. (2018) · 2018

Later among the works it cites.

The neural tangent kernel in high dimensions: Triple descent and a multi-scale theory of generalization

Adlam, B. and Pennington, J. (2020) · 2020

Later among the works it cites.