Interpretable Meta-Measure for Model Performance

Understand

Benchmarks for the evaluation of model performance play an important role in machine learning.

• However, there is no established way to describe and create new benchmarks.
• What is more, the most common benchmarks use performance measures that share several limitations.
• For example, the difference in performance for two models has no probabilistic interpretation, there is no reference point to indicate whether they represent a significant improvement, and it makes no sense to compare such differences between data sets.