Optimizing F-measure: A Tale of Two Approaches

Understand

F-measures are popular performance metrics, particularly for tasks with imbalanced data sets.

• Algorithms for learning to maximize F-measures follow two approaches: the empirical utility maximization (EUM) approach learns a classifier having optimal performance on training data, while the decision-theoretic approach learns a probabilistic model and then predicts labels with maximum expected F-measure.
• In this paper, we investigate the theoretical justifications and connections for these two approaches, and we study the conditions under which one approach is preferable to the other using synthetic and real datasets.
• Given accurate models, our results suggest that the two approaches are asymptotically equivalent given large training and test sets.