MCMC Using Ensembles of States for Problems with Fast and Slow Variables such as Gaussian Process Regression

Understand

I introduce a Markov chain Monte Carlo (MCMC) scheme in which sampling from a distribution with density pi(x) is done using updates operating on an "ensemble" of states.

• The current state x is first stochastically mapped to an ensemble, x^{(1)},...,x^{(K)}.
• This ensemble is then updated using MCMC updates that leave invariant a suitable ensemble density, rho(x^{(1)},...,x^{(K)}), defined in terms of pi(x^{(i)}) for i=1,...,K.
• Finally a single state is stochastically selected from the ensemble after these updates.