Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetry

Understand

Bayesian inference in deep neural networks is challenging due to the high-dimensional, strongly multi-modal parameter posterior density landscape.

• Markov chain Monte Carlo approaches asymptotically recover the true posterior but are considered prohibitively expensive for large modern architectures.
• Local methods, which have emerged as a popular alternative, focus on specific parameter regions that can be approximated by functions with tractable integrals.
• While these often yield satisfactory empirical results, they fail, by definition, to account for the multi-modality of the parameter posterior.