The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective

Understand

As various post hoc explanation methods are increasingly being leveraged to explain complex models in high-stakes settings, it becomes critical to develop a deeper understanding of whether and when the explanations output by these methods disagree with each other, and how such disagreements are resolved in practice.

• However, there is little to no research that provides answers to these critical questions.
• In this work, we formalize and study the disagreement problem in explainable machine learning.
• More specifically, we define the notion of disagreement between explanations, analyze how often such disagreements occur in practice, and how practitioners resolve these disagreements.