Evaluating and Calibrating Uncertainty Prediction in Regression Tasks

Understand

Predicting not only the target but also an accurate measure of uncertainty is important for many machine learning applications and in particular safety-critical ones.

• In this work we study the calibration of uncertainty prediction for regression tasks which often arise in real-world systems.
• We show that the existing definition for calibration of a regression uncertainty [Kuleshov et al.
• 2018] has severe limitations in distinguishing informative from non-informative uncertainty predictions.