Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance

Understand

Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering.

• This task may be cast as an optimization problem over all probability measures, and an initial distribution can be evolved to the desired minimizer dynamically via gradient flows.
• Mean-field models, whose law is governed by the gradient flow in the space of probability measures, may also be identified; particle approximations of these mean-field models form the basis of algorithms.
• The gradient flow approach is also the basis of algorithms for variational inference, in which the optimization is performed over a parameterized family of probability distributions such as Gaussians, and the underlying gradient flow is restricted to the parameterized family.