Repulsive Deep Ensembles are Bayesian

Understand

Deep ensembles have recently gained popularity in the deep learning community for their conceptual simplicity and efficiency.

• However, maintaining functional diversity between ensemble members that are independently trained with gradient descent is challenging.
• This can lead to pathologies when adding more ensemble members, such as a saturation of the ensemble performance, which converges to the performance of a single model.
• Moreover, this does not only affect the quality of its predictions, but even more so the uncertainty estimates of the ensemble, and thus its performance on out-of-distribution data.