A Compact Representation for Bayesian Neural Networks By Removing Permutation Symmetry

On the algebraic structure of feedforward network weight spaces

Robert Hecht-Nielsen · 1990

Earlier work this paper cites.

Basic algebra

Anthony W Knapp · 2007

Earlier work this paper cites.

Mcmc using hamiltonian dynamics

Radford M Neal et al · 2011

Earlier work this paper cites.

Bayesian learning via stochastic gradient langevin dynamics

Max Welling and Yee W Teh · 2011

Earlier work this paper cites.

Weight uncertainty in neural network

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra · 2015

Earlier work this paper cites.

Dropout as a bayesian approximation: Representing model uncertainty in deep learning

Yarin Gal and Zoubin Ghahramani · 2016

Earlier work this paper cites.

Simple and scalable predictive uncertainty estimation using deep ensembles

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell · 2017

Earlier work this paper cites.

Improving the identifiability of neural networks for bayesian inference

Arya A Pourzanjani, Richard M Jiang, and Linda R Petzold · 2017

Cited alongside, same era.

A scalable laplace approximation for neural networks

Hippolyt Ritter, Aleksandar Botev, and David Barber · 2018

Cited alongside, same era.

Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape

Original

Johanni Brea, Berfin Simsek, Bernd Illing, and Wulfram Gerstner · 2019

Cited alongside, same era.

Linear mode connectivity and the lottery ticket hypothesis

Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin · 2020

Cited alongside, same era.

Being bayesian, even just a bit, fixes overconfidence in relu networks

Agustinus Kristiadi, Matthias Hein, and Philipp Hennig · 2020

Cited alongside, same era.

On symmetries in variational bayesian neural nets

Richard Kurle, Tim Januschowski, Jan Gasthaus, and Yuyang Bernie Wang · 2021

Later among the works it cites.

The role of permutation invariance in linear mode connectivity of neural networks

Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur · 2022

Later among the works it cites.

Post-training neural network compression with variational bayesian quantization

Zipei Tan and Robert Bamler · 2022

Later among the works it cites.

Evaluating approximate inference in bayesian deep learning

Andrew Gordon Wilson, Pavel Izmailov, Matthew D Hoffman, Yarin Gal, Yingzhen Li, Melanie F Pradier, Sharad Vikram, Andrew Foong, Sanae Lotfi, and Sebastian Farquhar · 2022

Later among the works it cites.

Git re-basin: Merging models modulo permutation symmetries

Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa · 2023

Closest in time.

Towards efficient mcmc sampling in bayesian neural networks by exploiting symmetry