Selection by Prediction with Conformal p-values

Understand

Decision making or scientific discovery pipelines such as job hiring and drug discovery often involve multiple stages: before any resource-intensive step, there is often an initial screening that uses predictions from a machine learning model to shortlist a few candidates from a large pool.

• We study screening procedures that aim to select candidates whose unobserved outcomes exceed user-specified values.
• We develop a method that wraps around any prediction model to produce a subset of candidates while controlling the proportion of falsely selected units.
• Building upon the conformal inference framework, our method first constructs p-values that quantify the statistical evidence for large outcomes; it then determines the shortlist by comparing the p-values to a threshold introduced in the multiple testing literature.