2019

The Explanation Game: Explaining Machine Learning Models Using Shapley Values

Merrick, Luke, Taly, Ankur

Understand

A number of techniques have been proposed to explain a machine learning model's prediction by attributing it to the corresponding input features.

Popular among these are techniques that apply the Shapley value method from cooperative game theory.
While existing papers focus on the axiomatic motivation of Shapley values, and efficient techniques for computing them, they offer little justification for the game formulations used, and do not address the uncertainty implicit in their methods' outputs.
For instance, the popular SHAP algorithm's formulation may give substantial attributions to features that play no role in the model.

Built on

Shapley, L.S.: A value for n-person games. Contributions to the Theory of Games 2
1953
Earlier work this paper cites.
Young, H.P.: Monotonic solutions of cooperative games. International Journal of Game Theory 14
Original
1985
Earlier work this paper cites.
Efron, B., Tibshirani, R.: Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistal Science pp. 54–75 (1986)
1986
Earlier work this paper cites.
Kahneman, D., Miller, D.T.: Norm theory: Comparing reality to its alternatives. Psychological review 93
1986
Earlier work this paper cites.
Hesslow, G.: The problem of causal selection. In: Hilton, D.J. (ed.) Contemporary Science and Natural Explanation: Commonsense Conceptions of Causality. New York University Press (1988)
1988
Earlier work this paper cites.
Lipton, P.: Contrastive explanation. Royal Institute of Philosophy Supplement 27
1990
Earlier work this paper cites.
Cohen, S., Ruppin, E., Dror, G.: Feature selection based on the shapley value. In other words 1
2005
Earlier work this paper cites.
Hitchcock, C., Knobecaus, J.: Cause and norm. Journal of Philosophy 106
2009
Earlier work this paper cites.
Štrumbelj, E., Kononenko, I.: An efficient explanation of individual classifications using game theory. Journal of Machine Learning Research 11
2010
Earlier work this paper cites.
Maleki, S., Tran-Thanh, L., Hines, G., Rahwan, T., Rogers, A.: Bounding the estimation error of sampling-based shapley value approximation. arXiv preprint arXiv:1306.4265 (2013)
Original
2013
Earlier work this paper cites.

Similar

Štrumbelj, E., Kononenko, I.: Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems 41
2014
Cited alongside, same era.
Datta, A., Sen, S., Zick, Y.: Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In: 2016 IEEE symposium on security and privacy (SP). pp. 598–617. IEEE (2016)
2016
Cited alongside, same era.
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: Explaining the predictions of any classifier. In: SIGKDD international conference on knowledge discovery and data mining. pp. 1135–1144 (2016)
2016
Cited alongside, same era.
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T.Y.: Lightgbm: A highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems. pp. 3146–3154 (2017)
2017
Cited alongside, same era.
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems. pp. 4765–4774 (2017)
2017
Cited alongside, same era.
Miller, T.: Explanation in artificial intelligence: Insights from the social sciences. arXiv preprint arXiv:1706.07269 (2017)
Original
2017
Cited alongside, same era.
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: 34th International Conference on Machine Learning-Volume 70. pp. 3145–3153 (2017)
2017
Cited alongside, same era.
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70. pp. 3319–3328. JMLR. org (2017)
2017
Cited alongside, same era.
Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for deep neural networks. In: International Conference on Learning Representations (2018)
2018
Cited alongside, same era.
Chen, J., Song, L., Wainwright, M.J., Jordan, M.I.: L-shapley and c-shapley: Efficient model interpretation for structured data. arXiv preprint arXiv:1808.02610 (2018)
Original
2018
Cited alongside, same era.

Then

Dhurandhar, A., Chen, P., Luss, R., Tu, C., Ting, P., Shanmugam, K., Das, P.: Explanations based on the missing: Towards contrastive explanations with pertinent negatives. CoRR (2018), http://arxiv.org/abs/1802.07623
Original
2018
Later among the works it cites.
Lundberg, S.M., Erion, G.G., Lee, S.I.: Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888 (2018)
Original
2018
Later among the works it cites.
Aas, K., Jullum, M., Løland, A.: Explaining individual predictions when features are dependent: More accurate approximations to shapley values. arXiv preprint arXiv:1903.10464 (2019)
Original
2019
Closest in time.
Ancona, M., Oztireli, C., Gross, M.: Explaining deep neural networks with a polynomial time algorithm for shapley value approximation. In: Proceedings of the 36th International Conference on Machine Learning (2019)
2019
Closest in time.
Ghorbani, A., Zou, J.: Data shapley: Equitable valuation of data for machine learning. In: Proceedings of the 36th International Conference on Machine Learning (2019)
2019
Closest in time.
Holzinger, A., Kickmeier-Rust, M., Müller, H.: KANDINSKY Patterns as IQ-Test for Machine Learning. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) Machine Learning and Knowledge Extraction. pp. 1–14. Lecture Notes in Computer Science, Springer International Publishing, Cham (2019)
2019
Closest in time.
Hunt, X.J., Abbey, R., Tharrington, R., Huiskens, J., Wesdorp, N.: An ai-augmented lesion detection framework for liver metastases with model interpretability. arXiv preprint arXiv:1907.07713 (2019)
Original
2019
Closest in time.
Janzing, D., Minorics, L., Blöbaum, P.: Feature relevance quantification in explainable ai: A causal problem. arXiv preprint arXiv:1910.13413 (2019)
Original
2019
Closest in time.
Mittelstadt, B., Russell, C., Wachter, S.: Explaining explanations in ai. In: Proceedings of the conference on fairness, accountability, and transparency. pp. 279–288. ACM (2019)
2019
Closest in time.
Sundararajan, M., Najmi, A.: The many shapley values for model explanation. arXiv preprint arXiv:1908.08474 (2019)
Original
2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…