Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding

Understand

Predictive algorithms inform consequential decisions in settings with selective labels: outcomes are observed only for units selected by past decision makers.

• This creates an identification problem under unobserved confounding -- when selected and unselected units differ in unobserved ways that affect outcomes.
• We propose a framework for robust design and evaluation of predictive algorithms that bounds how much outcomes may differ between selected and unselected units with the same observed characteristics.
• These bounds formalize common empirical strategies including proxy outcomes and instrumental variables.