Estimating Uncertainty Online Against an Adversary

Understand

Assessing uncertainty is an important step towards ensuring the safety and reliability of machine learning systems.

• Existing uncertainty estimation techniques may fail when their modeling assumptions are not met, e.g.
• when the data distribution differs from the one seen at training time.
• Here, we propose techniques that assess a classification algorithm's uncertainty via calibrated probabilities (i.e.