The geometry of nonlinear least squares with applications to sloppy models and optimization

Understand

Parameter estimation by nonlinear least squares minimization is a common problem with an elegant geometric interpretation: the possible parameter values of a model induce a manifold in the space of data predictions.

• The minimization problem is then to find the point on the manifold closest to the data.
• We show that the model manifolds of a large class of models, known as sloppy models, have many universal features; they are characterized by a geometric series of widths, extrinsic curvatures, and parameter-effects curvatures.
• A number of common difficulties in optimizing least squares problems are due to this common structure.