Training independent subnetworks for robust prediction

Understand

Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.

• However, these methods still require multiple forward passes for prediction, leading to a significant computational cost.
• In this work, we show a surprising result: the benefits of using multiple predictions can be achieved `for free' under a single model's forward pass.
• In particular, we show that, using a multi-input multi-output (MIMO) configuration, one can utilize a single model's capacity to train multiple subnetworks that independently learn the task at hand.