A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping

Understand

A survey of existing methods for stopping active learning (AL) reveals the needs for methods that are: more widely applicable; more aggressive in saving annotations; and more stable across changing datasets.

• A new method for stopping AL based on stabilizing predictions is presented that addresses these needs.
• Furthermore, stopping methods are required to handle a broad range of different annotation/performance tradeoff valuations.
• Despite this, the existing body of work is dominated by conservative methods with little (if any) attention paid to providing users with control over the behavior of stopping methods.