Optimally-Weighted Herding is Bayesian Quadrature

Understand

Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution.

• A related task is choosing samples for estimating integrals using Bayesian quadrature.
• We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature.
• We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method.