No Free Lunch for Approximate MCMC

Understand

It is widely known that the performance of Markov chain Monte Carlo (MCMC) can degrade quickly when targeting computationally expensive posterior distributions, such as when the sample size is large.

• This has motivated the search for MCMC variants that scale well to large datasets.
• One popular general approach has been to look at only a subsample of the data at every step.
• In this note, we point out that well-known MCMC convergence results often imply that these ``subsampling'' MCMC algorithms cannot greatly improve performance.