LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning

Understand

Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive.

• To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be label-efficient: achieving high predictive performance from relatively few labeled examples.
• While obtaining the best label-efficiency in practice often requires combinations of these techniques, existing benchmark and evaluation frameworks do not capture a concerted combination of all such techniques.
• This paper addresses this deficiency by introducing LabelBench, a new computationally-efficient framework for joint evaluation of multiple label-efficient learning techniques.