Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks

Understand

Algorithmic bias is of increasing concern, both to the research community, and society at large.

• Bias in AI is more abstract and unintuitive than traditional forms of discrimination and can be more difficult to detect and mitigate.
• A clear gap exists in the current literature on evaluating the relative bias in the performance of multi-class classifiers.
• In this work, we propose two simple yet effective metrics, Combined Error Variance (CEV) and Symmetric Distance Error (SDE), to quantitatively evaluate the class-wise bias of two models in comparison to one another.