NOMU: Neural Optimization-based Model Uncertainty

Understand

We study methods for estimating model uncertainty for neural networks (NNs) in regression.

• To isolate the effect of model uncertainty, we focus on a noiseless setting with scarce training data.
• We introduce five important desiderata regarding model uncertainty that any method should satisfy.
• However, we find that established benchmarks often fail to reliably capture some of these desiderata, even those that are required by Bayesian theory.