Approaching Peak Ground Truth

Understand

Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such.

• Especially in the biomedical domain, annotations are subjective and suffer from low inter- and intra-rater reliability.
• Since annotations only reflect one interpretation of the real world, this can lead to sub-optimal predictions even though the model achieves high similarity scores.
• Here, the theoretical concept of PGT is introduced.