Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation

Understand

Commonly used evaluation measures including Recall, Precision, F-Measure and Rand Accuracy are biased and should not be used without clear understanding of the biases, and corresponding identification of chance or base case levels of the statistic.

• Using these measures a system that performs worse in the objective sense of Informedness, can appear to perform better under any of these commonly used measures.
• We discuss several concepts and measures that reflect the probability that prediction is informed versus chance.
• Informedness and introduce Markedness as a dual measure for the probability that prediction is marked versus chance.