Forks Over Knives: Predictive Inconsistency in Criminal Justice Algorithmic Risk Assessment Tools

Understand

Big data and algorithmic risk prediction tools promise to improve criminal justice systems by reducing human biases and inconsistencies in decision making.

• Yet different, equally-justifiable choices when developing, testing, and deploying these sociotechnical tools can lead to disparate predicted risk scores for the same individual.
• Synthesizing diverse perspectives from machine learning, statistics, sociology, criminology, law, philosophy and economics, we conceptualize this phenomenon as predictive inconsistency.
• We describe sources of predictive inconsistency at different stages of algorithmic risk assessment tool development and deployment and consider how future technological developments may amplify predictive inconsistency.