Understand
We explore the problem of learning under selective labels in the context of algorithm-assisted decision making.
- Selective labels is a pervasive selection bias problem that arises when historical decision making blinds us to the true outcome for certain instances.
- Examples of this are common in many applications, ranging from predicting recidivism using pre-trial release data to diagnosing patients.
- In this paper we discuss why selective labels often cannot be effectively tackled by standard methods for adjusting for sample selection bias, even if there are no unobservables.
Built on
Clinical versus statistical prediction: A theoretical analysis and a review of the evidence
Meehl, Paul E · 1954
Earlier work this paper cites.
Clinical versus actuarial judgment
Dawes, Robyn M, Faust, David, and Meehl, Paul E · 1989
Earlier work this paper cites.
Competence in experts: The role of task characteristics
Shanteau, James · 1992
Earlier work this paper cites.
Clinical versus mechanical prediction: a meta-analysis
Grove, William M, Zald, David H, Lebow, Boyd S, Snitz, Beth E, and Nelson, Chad · 2000
Earlier work this paper cites.
Learning and evaluating classifiers under sample selection bias
Zadrozny, Bianca · 2004
Earlier work this paper cites.
Similar
Correcting sample selection bias by unlabeled data
Huang, Jiayuan, Gretton, Arthur, Borgwardt, Karsten M, Schölkopf, Bernhard, and Smola, Alex J · 2007
Cited alongside, same era.
Review of inverse probability weighting for dealing with missing data
Seaman, Shaun R and White, Ian R · 2013
Cited alongside, same era.
Statistical analysis with missing data
Little, Roderick JA and Rubin, Donald B · 2014
Cited alongside, same era.
Why task domains (still) matter for understanding expertise
Shanteau, James · 2015
Cited alongside, same era.
Learning with rejection
Cortes, Corinna, DeSalvo, Giulia, and Mohri, Mehryar · 2016
Cited alongside, same era.
Then
Bail Comissioner Handbook
New Hampshire Judicial Branch · 2016
Later among the works it cites.
Human decisions and machine predictions
Kleinberg, Jon, Lakkaraju, Himabindu, Leskovec, Jure, Ludwig, Jens, and Mullainathan, Sendhil · 2017
Later among the works it cites.
The selective labels problem: Evaluating algorithmic predictions in the presence of unobservables
Lakkaraju, Himabindu, Kleinberg, Jon, Leskovec, Jure, Ludwig, Jens, and Mullainathan, Sendhil · 2017
Later among the works it cites.
Predict responsibly: Increasing fairness by learning to defer
Madras, David, Pitassi, Toniann, and Zemel, Richard · 2017
Later among the works it cites.
Residual unfairness in fair machine learning from prejudiced data
Kallus, Nathan and Zhou, Angela · 2018
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…