Wide Mean-Field Bayesian Neural Networks Ignore the Data

Understand

Bayesian neural networks (BNNs) combine the expressive power of deep learning with the advantages of Bayesian formalism.

• In recent years, the analysis of wide, deep BNNs has provided theoretical insight into their priors and posteriors.
• However, we have no analogous insight into their posteriors under approximate inference.
• In this work, we show that mean-field variational inference entirely fails to model the data when the network width is large and the activation function is odd.