Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting

Understand

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) approach that exhibits favourable exploration properties in high-dimensional models such as neural networks.

• Unfortunately, HMC has limited use in large-data regimes and little work has explored suitable approaches that aim to preserve the entire Hamiltonian.
• In our work, we introduce a new symmetric integration scheme for split HMC that does not rely on stochastic gradients.
• We show that our new formulation is more efficient than previous approaches and is easy to implement with a single GPU.