Failure Detection in Medical Image Classification: A Reality Check and Benchmarking Testbed

Understand

Failure detection in automated image classification is a critical safeguard for clinical deployment.

• Detected failure cases can be referred to human assessment, ensuring patient safety in computer-aided clinical decision making.
• Despite its paramount importance, there is insufficient evidence about the ability of state-of-the-art confidence scoring methods to detect test-time failures of classification models in the context of medical imaging.
• This paper provides a reality check, establishing the performance of in-domain misclassification detection methods, benchmarking 9 widely used confidence scores on 6 medical imaging datasets with different imaging modalities, in multiclass and binary classification settings.