Why rankings of biomedical image analysis competitions should be interpreted with care

Understand

International challenges have become the standard for validation of biomedical image analysis methods.

• Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed.
• In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now.
• We demonstrate the importance of challenges and show that the lack of quality control has critical consequences.