DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation

Understand

Deep ensembles perform better than a single network thanks to the diversity among their members.

• Recent approaches regularize predictions to increase diversity; however, they also drastically decrease individual members' performances.
• In this paper, we argue that learning strategies for deep ensembles need to tackle the trade-off between ensemble diversity and individual accuracies.
• Motivated by arguments from information theory and leveraging recent advances in neural estimation of conditional mutual information, we introduce a novel training criterion called DICE: it increases diversity by reducing spurious correlations among features.