2020

Aligning Faithful Interpretations with their Social Attribution

Jacovi, Alon, Goldberg, Yoav

Understand

We find that the requirement of model interpretations to be faithful is vague and incomplete.

With interpretation by textual highlights as a case-study, we present several failure cases.
Borrowing concepts from social science, we identify that the problem is a misalignment between the causal chain of decisions (causal attribution) and the attribution of human behavior to the interpretation (social attribution).
We re-formulate faithfulness as an accurate attribution of causality to the model, and introduce the concept of aligned faithfulness: faithful causal chains that are aligned with their expected social behavior.

Built on

Contrastive algorithmic fairness: Part 1 (theory)
Original
Tapabrata Chakraborti, Arijit Patra, and J. Alison Noble. 2019 · 1905
Earlier work this paper cites.
Do transformer attention heads provide transparency in abstractive summarization?
Original
Joris Baan, Maartje ter Hoeve, Marlies van der Wees, Anne Schuth, and Maarten de Rijke. 2019 · 1907
Earlier work this paper cites.
Learning to deceive with attention-based explanations
Original
Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, and Zachary C. Lipton. 2019 · 1909
Earlier work this paper cites.
Deriving machine attention from human rationales
Yujia Bao, Shiyu Chang, Mo Yu, and Regina Barzilay. 2018 · 1913
Earlier work this paper cites.
An experimental study of apparent behavior
Fritz Heider and Marianne Simmel. 1944 · 1944
Earlier work this paper cites.
The psychology of interpersonal relations
Fritz Heider, American Psychological Association, and Inc Ovid Technologies. 1958 · 1958
Earlier work this paper cites.
Logic and causal attribution
Denis J Hilton. 1988 · 1986
Earlier work this paper cites.
Knowledge-based causal attribution: The abnormal conditions focus model
Denis J Hilton and Ben R Slugoski. 1986 · 1986
Earlier work this paper cites.
The problem of causal selection
Germund Hesslow. 1988 · 1988
Earlier work this paper cites.
Contrastive explanation
Peter Lipton. 1990 · 1990
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J Williams. 1992 · 1992
Earlier work this paper cites.
Contrastive and counterfactual thinking in causal judgment
Ann Mcgill and Jill Klein. 1993 · 1993
Earlier work this paper cites.
Problems with shapley-value-based explanations as feature importance measures
Original
I. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle A. Friedler. 2020 · 2002
Earlier work this paper cites.
Invariant rationalization
Original
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi S. Jaakkola. 2020 · 2003
Earlier work this paper cites.
Towards transparent robotic planning via contrastive explanations
Original
Sheng-Hui Chen, Kayla Boggess, and Lu Feng. 2020 · 2003
Earlier work this paper cites.
Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?
Original
Alon Jacovi and Yoav Goldberg. 2020 · 2004
Earlier work this paper cites.
Causalm: Causal model explanation through counterfactual language models
Original
Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. 2020 · 2005
Earlier work this paper cites.

Similar

Evaluating explainable AI: which algorithmic explanations help users predict model behavior?
Original
Peter Hase and Mohit Bansal. 2020 · 2005
Cited alongside, same era.
The psychology of counterfactual thinking
Denis J Hilton, 1966 Mandel, David R., and Patrizia Catellani. 2005 · 2005
Cited alongside, same era.
Learning to faithfully rationalize by construction
Original
Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, and Byron C. Wallace. 2020 · 2005
Cited alongside, same era.
Obtaining faithful interpretations from compositional neural networks
Original
Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, and Matt Gardner. 2020 · 2005
Cited alongside, same era.
The structure and function of explanations
Tania Lombrozo. 2006 · 2006
Cited alongside, same era.
Using "annotator rationales" to improve machine learning for text categorization
Omar Zaidan, Jason Eisner, and Christine D. Piatko. 2007 · 2007
Cited alongside, same era.
The hazards of explanation: Overgeneralization in the face of exceptions
Joseph Jay Williams, Tania Lombrozo, and Bob Rehder. 2013 · 2013
Cited alongside, same era.
Auto-encoding variational bayes
Original
Diederik P. Kingma and Max Welling. 2014 · 2014
Cited alongside, same era.
Learning to compose neural networks for question answering
Original
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016 · 2016
Cited alongside, same era.
Rationalizing neural predictions
Original
Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. 2016 · 2016
Cited alongside, same era.
“Why Should I Trust You?”: Explaining the predictions of any classifier
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016 · 2016
Cited alongside, same era.
Contrastive constraints guide explanation-based category learning
Seth Chin-Parker and Julie Cantelon. 2017 · 2017
Cited alongside, same era.

Then

BERT: pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Later among the works it cites.
Explanations based on the missing: Towards contrastive explanations with pertinent negatives
Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Pai-Shun Ting, Karthikeyan Shanmugam, and Payel Das. 2018 · 2018
Later among the works it cites.
Pathologies of neural models make interpretation difficult
Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan L. Boyd-Graber. 2018 · 2018
Later among the works it cites.
The mythos of model interpretability
Zachary C. Lipton. 2018 · 2018
Later among the works it cites.
Interpretable neural predictions with differentiable binary variables
Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2019 · 2019
Later among the works it cites.
A game theoretic approach to class-wise selective rationalization
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi S. Jaakkola. 2019 · 2019
Later among the works it cites.
Eraser: A benchmark to evaluate rationalized nlp models
Original
Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2019 · 2019
Later among the works it cites.
The (un)reliability of saliency methods
Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. 2019 · 2019
Later among the works it cites.
Explanation in artificial intelligence: Insights from the social sciences
Tim Miller. 2019 · 2019
Later among the works it cites.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
Cynthia Rudin. 2019 · 2019
Later among the works it cites.
Is attention interpretable?
Sofia Serrano and Noah A. Smith. 2019 · 2019
Later among the works it cites.
Evaluating gender bias in machine translation
Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019 · 2019
Later among the works it cites.
Attention is not not explanation
Sarah Wiegreffe and Yuval Pinter. 2019 · 2019
Later among the works it cites.
Leveraging rationales to improve human task performance
Devleena Das and Sonia Chernova. 2020 · 2020
Closest in time.
Why attention is not explanation: Surgical intervention and causal reasoning about neural models
Christopher Grimsley, Elijah Mayfield, and Julia R.S. Bursten. 2020 · 2020
Closest in time.
Explainable reinforcement learning through a causal lens
Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2020 · 2020
Closest in time.
Contrastive explanation: A structural-model approach
Original
Tim Miller. 2020 · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…