Calibration of Neural Networks

Understand

Neural networks solving real-world problems are often required not only to make accurate predictions but also to provide a confidence level in the forecast.

• The calibration of a model indicates how close the estimated confidence is to the true probability.
• This paper presents a survey of confidence calibration problems in the context of neural networks and provides an empirical comparison of calibration methods.
• We analyze problem statement, calibration definitions, and different approaches to evaluation: visualizations and scalar measures that estimate whether the model is well-calibrated.