Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)

Understand

We investigate learning of the differential geometric structure of a data manifold embedded in a high-dimensional Euclidean space.

• We first analyze kernel-based algorithms and show that under the usual regularizations, non-probabilistic methods cannot recover the differential geometric structure, but instead find mostly linear manifolds or spaces equipped with teleports.
• To properly learn the differential geometric structure, non-probabilistic methods must apply regularizations that enforce large gradients, which go against common wisdom.
• We repeat the analysis for probabilistic methods and find that under reasonable priors, the geometric structure can be recovered.