Confident Learning: Estimating Uncertainty in Dataset Labels

Understand

Learning exists in the context of data, yet notions of confidence typically focus on model predictions, not label quality.

• Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence.
• Whereas numerous studies have developed these principles independently, here, we combine them, building on the assumption of a class-conditional noise process to directly estimate the joint distribution between noisy (given) labels and uncorrupted (unknown) labels.
• This results in a generalized CL which is provably consistent and experimentally performant.