Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability

Understand

Algorithms are increasingly used to aid, or in some cases supplant, human decision-making, particularly for decisions that hinge on predictions.

• As a result, two additional features in addition to prediction quality have generated interest: (i) to facilitate human interaction and understanding with these algorithms, we desire prediction functions that are in some fashion simple or interpretable; and (ii) because they influence consequential decisions, we also want them to produce equitable allocations.
• We develop a formal model to explore the relationship between the demands of simplicity and equity.
• Although the two concepts appear to be motivated by qualitatively distinct goals, we show a fundamental inconsistency between them.