2020

Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations

Sreedharan, Sarath, Soni, Utkarsh, Verma, Mudit et al.

Understand

As increasingly complex AI systems are introduced into our daily lives, it becomes important for such systems to be capable of explaining the rationale for their decisions and allowing users to contest these decisions.

A significant hurdle to allowing for such explanatory dialogue could be the vocabulary mismatch between the user and the AI system.
This paper introduces methods for providing contrastive explanations in terms of user-specified concepts for sequential decision-making settings where the system's model of the task may be best represented as an inscrutable model.
We do this by building partial symbolic models of a local approximation of the task that can be leveraged to answer the user queries.

Built on

The problem of causal selection
Germund Hesslow · 1988
Earlier work this paper cites.
Learning by experimentation: The operator refinement method
Jaime G Carbonell and Yolanda Gil · 1990
Earlier work this paper cites.
Pddl-the planning domain definition language
Drew McDermott, Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin Ram, Manuela Veloso, Daniel Weld, and David Wilkins · 1998
Earlier work this paper cites.
A short introduction to boosting
Yoav Freund, Robert Schapire, and Naoki Abe · 1999
Earlier work this paper cites.
Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning
Richard S Sutton, Doina Precup, and Satinder Singh · 1999
Earlier work this paper cites.
How the mind explains behavior: Folk explanations, meaning, and social interaction
Bertram F Malle · 2006
Earlier work this paper cites.
Arms: An automatic knowledge engineering tool for learning action models for ai planning
Kangheng Wu, Qiang Yang, and Yunfei Jiang · 2007
Earlier work this paper cites.
Do you get it? user-evaluated explainable bdi agents
Joost Broekens, Maaike Harbers, Koen Hindriks, Karel Van Den Bosch, Catholijn Jonker, and John-Jules Meyer · 2010
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Earlier work this paper cites.
A concise introduction to models and methods for automated planning
Hector Geffner and Blai Bonet · 2013
Earlier work this paper cites.
urandom(4) - Linux man page , March 2013
Linux · 2013
Earlier work this paper cites.
Prioritized experience replay
Original
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver · 2015
Earlier work this paper cites.
Openai gym
Original
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba · 2016
Earlier work this paper cites.
Why should i trust you?: Explaining the predictions of any classifier
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin · 2016
Earlier work this paper cites.
Grad-cam: Why did you say that?
Original
Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra · 2016
Earlier work this paper cites.
Network dissection: Quantifying interpretability of deep visual representations
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba · 2017
Earlier work this paper cites.

Similar

Plan explanations as model reconciliation: Moving beyond explanation as soliloquy
Tathagata Chakraborti, Sarath Sreedharan, Yu Zhang, and Subbarao Kambhampati · 2017
Cited alongside, same era.
Improving robot controller transparency through autonomous policy explanation
Bradley Hayes and Julie A Shah · 2017
Cited alongside, same era.
Efficient, safe, and probably approximately complete learning of action models
Original
Roni Stern and Brendan Juba · 2017
Cited alongside, same era.
Axiomatic attribution for deep networks
Mukund Sundararajan, Ankur Taly, and Qiqi Yan · 2017
Cited alongside, same era.
Debugging agent programs with why? questions
Michael Winikoff · 2017
Cited alongside, same era.
Visualizing and Understanding Atari Agents
Samuel Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern · 2018
Cited alongside, same era.
Interactive naming for explaining deep neural networks: a formative study
Original
Mandana Hamidi-Haines, Zhongang Qi, Alan Fern, Fuxin Li, and Prasad Tadepalli · 2018
Cited alongside, same era.
Metrics for explainable ai: Challenges and prospects
Original
Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman · 2018
Cited alongside, same era.
Transparency and explanation in deep reinforcement learning neural networks
Rahul Iyer, Yuezhang Li, Huao Li, Michael Lewis, Ramitha Sundar, and Katia Sycara · 2018
Cited alongside, same era.
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
B. Kim, Wattenberg M., J. Gilmer, Cai C., Wexler J., , F. Viegas, and R. Sayres · 2018
Cited alongside, same era.
Explanation in artificial intelligence: Insights from the social sciences
Tim Miller · 2018
Cited alongside, same era.
gym-sokoban
Max-Philipp B. Schrader · 2018
Cited alongside, same era.

Then

Human-centered tools for coping with imperfect algorithms during medical decision-making
Carrie J Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viegas, Greg S Corrado, Martin C Stumpe, et al · 2019
Later among the works it cites.
Towards automatic concept-based explanations
Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim · 2019
Later among the works it cites.
Explainable Reinforcement Learning via Reward Decomposition
Zoe Juozapaitis, Anurag Koul, Alan Fern, Martin Erwig, and Finale Doshi-Velez · 2019
Later among the works it cites.
Varieties of Explainable Agency
Pat Langley · 2019
Later among the works it cites.
Generating contrastive explanations with monotonic attribute functions
Original
Ronny Luss, Pin-Yu Chen, Amit Dhurandhar, Prasanna Sattigeri, Yunfeng Zhang, Karthikeyan Shanmugam, and Chun-Chen Tu · 2019
Later among the works it cites.
Explain your move: Understanding agent actions using specific and relevant feature attribution
Original
Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishnamurthy, and Sameer Singh · 2019
Later among the works it cites.
Reinforcement learning interpretation methods: A survey
Alnour Alharin, Thanh-Nam Doan, and Mina Sartipi · 2020
Closest in time.
The emerging landscape of explainable ai planning and decision making
Tathagata Chakraborti, Sarath Sreedharan, and Subbarao Kambhampati · 2020
Closest in time.
Concept bottleneck models
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang · 2020
Closest in time.
Contrastive explanations for reinforcement learning via embedded self predictions
Original
Zhengxian Lin, Kim-Ho Lam, and Alan Fern · 2020
Closest in time.
Explainable reinforcement learning through a causal lens
Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere · 2020
Closest in time.
Montezuma’s revenge (video game) — Wikipedia, the free encyclopedia, 2019
Wikipedia contributors · 2020
Closest in time.
Machine versus human attention in deep reinforcement learning tasks
Original
Ruohan Zhang, Sihang Guo, Bo Liu, Yifeng Zhu, Mary Hayhoe, Dana Ballard, and Peter Stone · 2020
Closest in time.
Symbols as a lingua franca for bridging human-ai chasm for explainable and advisable AI systems
Subbarao Kambhampati, Sarath Sreedharan, Mudit Verma, Yantian Zha, and Lin Guan · 2021
Closest in time.
Trust-aware planning: Modeling trust evolution in longitudinal human-robot interaction
Original
Zahra Zahedi, Mudit Verma, Sarath Sreedharan, and Subbarao Kambhampati · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…