Conformal Prediction Sets with Limited False Positives

Understand

We develop a new approach to multi-label conformal prediction in which we aim to output a precise set of promising prediction candidates with a bounded number of incorrect answers.

• Standard conformal prediction provides the ability to adapt to model uncertainty by constructing a calibrated candidate set in place of a single prediction, with guarantees that the set contains the correct answer with high probability.
• In order to obey this coverage property, however, conformal sets can become inundated with noisy candidates -- which can render them unhelpful in practice.
• This is particularly relevant to practical applications where there is a limited budget, and the cost (monetary or otherwise) associated with false positives is non-negligible.