Practical Obstacles to Deploying Active Learning

Understand

Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget.

• In AL one iteratively selects training examples for annotation, often those for which the current model is most uncertain (by some measure).
• The hope is that active sampling leads to better performance than would be achieved under independent and identically distributed (i.i.d.) random samples.
• While AL has shown promise in retrospective evaluations, these studies often ignore practical obstacles to its use.