2020

Explainability in Graph Neural Networks: A Taxonomic Survey

Yuan, Hao, Yu, Haiyang, Gui, Shurui et al.

Understand

Deep learning methods are achieving ever-increasing performance on many artificial intelligence tasks.

A major limitation of deep models is that they are not amenable to interpretability.
This limitation can be circumvented by developing post hoc techniques to explain the predictions, giving rise to the area of explainability.
Recently, explainability of deep models on images and texts has achieved significant progress.

Built on

H. W. Kuhn and A. W. Tucker, Contributions to the Theory of Games . Princeton University Press, 1953, vol. 2
1953
Earlier work this paper cites.
A. K. Debnath, R. L. Lopez de Compadre, G. Debnath, A. J. Shusterman, and C. Hansch, “Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity,” Journal of medicinal chemistry , vol. 34, no. 2, pp. 786–797, 1991
1991
Earlier work this paper cites.
R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” Advances in neural information processing systems , vol. 12, pp. 1057–1063, 1999
1999
Earlier work this paper cites.
D. Margaritis and S. Thrun, “Bayesian network induction via local neighborhoods,” Advances in neural information processing systems , vol. 12, pp. 505–511, 1999
1999
Earlier work this paper cites.
R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and U. Alon, “Network motifs: simple building blocks of complex networks,” Science , vol. 298, no. 5594, pp. 824–827, 2002
2002
Earlier work this paper cites.
R. Albert and A.-L. Barabási, “Statistical mechanics of complex networks,” Reviews of modern physics , vol. 74, no. 1, p. 47, 2002
2002
Earlier work this paper cites.
U. Alon, “Network motifs: theory and experimental approaches,” Nature Reviews Genetics , vol. 8, no. 6, pp. 450–461, 2007
2007
Earlier work this paper cites.
I. F. Martins, A. L. Teixeira, L. Pinheiro, and A. O. Falcao, “A bayesian approach to in silico blood-brain barrier penetration modeling,” Journal of chemical information and modeling , vol. 52, no. 6, pp. 1686–1697, 2012
2012
Earlier work this paper cites.
S. Ji, W. Xu, M. Yang, and K. Yu, “3d convolutional neural networks for human action recognition,” IEEE transactions on pattern analysis and machine intelligence , vol. 35, no. 1, pp. 221–231, 2013
2013
Earlier work this paper cites.
K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034 , 2013
Original
2013
Earlier work this paper cites.
R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” in Proceedings of the 2013 conference on empirical methods in natural language processing , 2013, pp. 1631–1642
2013
Earlier work this paper cites.
M. Yamada, W. Jitkrittum, L. Sigal, E. P. Xing, and M. Sugiyama, “High-dimensional feature selection by feature-wise kernelized lasso,” Neural computation , vol. 26, no. 1, pp. 185–207, 2014
2014
Earlier work this paper cites.
L. Dong, F. Wei, C. Tan, D. Tang, M. Zhou, and K. Xu, “Adaptive recursive neural network for target-dependent twitter sentiment classification,” in Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers) , 2014, pp. 49–54
2014
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations , 2015
2015
Earlier work this paper cites.
M. Henaff, J. Bruna, and Y. LeCun, “Deep convolutional networks on graph-structured data,” arXiv preprint arXiv:1506.05163 , 2015
Original
2015
Earlier work this paper cites.
S. Bach, A. Binder, G. Montavon, F. Klauschen, K. Müller, and W. Samek, “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,” PLoS ONE , vol. 10, no. 7, p. e0130140, 2015
2015
Earlier work this paper cites.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 2921–2929
2016
Earlier work this paper cites.
E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with gumbel-softmax,” in International Conference on Learning Representations , 2016
2016
Earlier work this paper cites.
M. T. Ribeiro, S. Singh, and C. Guestrin, “” why should i trust you?” explaining the predictions of any classifier,” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , 2016, pp. 1135–1144
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , 2017, pp. 5998–6008
2017
Earlier work this paper cites.
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in International Conference on Learning Representations , 2017
2017
Earlier work this paper cites.
D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg, “Smoothgrad: removing noise by adding noise,” arXiv preprint arXiv:1706.03825 , 2017
Original
2017
Earlier work this paper cites.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in 2017 IEEE International Conference on Computer Vision (ICCV) . IEEE, 2017, pp. 618–626
2017
Earlier work this paper cites.
P. Dabkowski and Y. Gal, “Real time image saliency for black box classifiers,” in Advances in Neural Information Processing Systems , 2017, pp. 6967–6976
2017
Earlier work this paper cites.
C. Olah, A. Mordvintsev, and L. Schubert, “Feature visualization,” Distill , 2017, https://distill.pub/2017/feature-visualization
2017
Earlier work this paper cites.
A. Shrikumar, P. Greenside, and A. Kundaje, “Learning important features through propagating activation differences,” in International Conference on Machine Learning , 2017, pp. 3145–3153
2017
Earlier work this paper cites.
C. Louizos, M. Welling, and D. P. Kingma, “Learning sparse neural networks through l _ 0 l\_0 regularization,” arXiv preprint arXiv:1712.01312 , 2017
Original
2017
Earlier work this paper cites.
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton et al. , “Mastering the game of go without human knowledge,” nature , vol. 550, no. 7676, pp. 354–359, 2017
2017
Earlier work this paper cites.
K. Maruhashi, M. Todoriki, T. Ohwa, K. Goto, Y. Hasegawa, H. Inakoshi, and H. Anai, “Learning multi-way relations via tensor decomposition with neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 32, no. 1, 2018
2018
Earlier work this paper cites.
C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, and A. Mordvintsev, “The building blocks of interpretability,” Distill , 2018, https://distill.pub/2018/building-blocks
2018
Earlier work this paper cites.

Similar

M. Du, N. Liu, Q. Song, and X. Hu, “Towards explanation of dnn-based prediction with guided feature inversion,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2018, pp. 1358–1367
2018
Cited alongside, same era.
F. K. Došilović, M. Brčić, and N. Hlupić, “Explainable artificial intelligence: A survey,” in 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO) . IEEE, 2018, pp. 0210–0215
2018
Cited alongside, same era.
M. Zhang, Z. Cui, M. Neumann, and Y. Chen, “An end-to-end deep learning architecture for graph classification.” in AAAI , vol. 18, 2018, pp. 4438–4445
2018
Cited alongside, same era.
M. Zhang and Y. Chen, “Link prediction based on graph neural networks,” in Advances in Neural Information Processing Systems , 2018, pp. 5165–5175
2018
Cited alongside, same era.
H. Gao, Z. Wang, and S. Ji, “Large-scale learnable graph convolutional networks,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2018, pp. 1416–1424
2018
Cited alongside, same era.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in International Conference on Learning Representations , 2018
2018
Cited alongside, same era.
K. K. Thekumparampil, C. Wang, S. Oh, and L.-J. Li, “Attention-based graph neural network for semi-supervised learning,” arXiv preprint arXiv:1803.03735 , 2018
Original
2018
Cited alongside, same era.
J. Chen, L. Song, M. J. Wainwright, and M. I. Jordan, “Learning to explain: An information-theoretic perspective on model interpretation,” in International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, vol. 80. PMLR, 2018, pp. 882–891
2018
Cited alongside, same era.
J. Chen, L. Song, M. Wainwright, and M. Jordan, “Learning to explain: An information-theoretic perspective on model interpretation,” in Proceedings of the 35th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. Stockholmsmässan, Stockholm Sweden: PMLR, 10–15 Jul 2018, pp. 883–892
2018
Cited alongside, same era.
M. Gardner, J. Grus, M. Neumann, O. Tafjord, P. Dasigi, N. Liu, M. Peters, M. Schmitz, and L. Zettlemoyer, “Allennlp: A deep semantic natural language processing platform,” arXiv preprint arXiv:1803.07640 , 2018
Original
2018
Cited alongside, same era.
Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande, “Moleculenet: a benchmark for molecular machine learning,” Chemical science , vol. 9, no. 2, pp. 513–530, 2018
2018
Cited alongside, same era.
K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?” in International Conference on Learning Representations , 2019
2019
Cited alongside, same era.

Then

H. Mao, X. Liu, N. Duffield, H. Yuan, S. Ji, and B. Mohanty, “Context-aware deep representation learning for geo-spatiotemporal analysis,” in 2020 IEEE International Conference on Data Mining (ICDM) . IEEE, 2020, pp. 392–401
2020
Closest in time.
H. Yuan, L. Cai, X. Hu, J. Wang, and S. Ji, “Interpreting image classifiers by generating discrete masks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020
2020
Closest in time.
Z. Wang, X. Hu, and S. Ji, “iCapsNets: Towards interpretable capsule networks for text classification,” arXiv preprint arXiv:2006.00075 , 2020
Original
2020
Closest in time.
A. Rai, “Explainable ai: From black box to glass box,” Journal of the Academy of Marketing Science , vol. 48, no. 1, pp. 137–141, 2020
2020
Closest in time.
Y. Ma and J. Tang, Deep Learning on Graphs . Cambridge University Press, 2020
2020
Closest in time.
M. Liu, H. Gao, and S. Ji, “Towards deeper graph neural networks,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2020, pp. 338–348
2020
Closest in time.
L. Cai and S. Ji, “A multi-scale approach for graph link prediction,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 04, 2020, pp. 3308–3315
2020
Closest in time.
M. Liu, Z. Wang, and S. Ji, “Non-local graph neural networks,” arXiv preprint arXiv:2005.14612 , 2020
Original
2020
Closest in time.
H. Yuan and S. Ji, “Structpool: Structured graph pooling via conditional random fields,” in International Conference on Learning Representations , 2020
2020
Closest in time.
Z. Wang and S. Ji, “Second-order pooling for graph neural networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020
2020
Closest in time.
H. Yuan, J. Tang, X. Hu, and S. Ji, “XGNN: Towards model-level explanations of graph neural networks,” ser. KDD ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 430–438. [Online]. Available: https://doi.org/10.1145/3394486.3403085
2020
Closest in time.
D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang, “Parameterized explainer for graph neural network,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 19 620–19 631
2020
Closest in time.
Q. Huang, M. Yamada, Y. Tian, D. Singh, D. Yin, and Y. Chang, “Graphlime: Local interpretable model explanations for graph neural networks,” arXiv preprint arXiv:2001.06216 , 2020
Original
2020
Closest in time.
Y. Zhang, D. Defazio, and A. Ramesh, “Relex: A model-agnostic relational model explainer,” arXiv preprint arXiv:2006.00305 , 2020
Original
2020
Closest in time.
M. Vu and M. T. Thai, “Pgm-explainer: Probabilistic graphical model explanations for graph neural networks,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 12 225–12 235
2020
Closest in time.
A. Jacovi and Y. Goldberg, “Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Online: Association for Computational Linguistics, Jul. 2020, pp. 4198–4205
2020
Closest in time.
B. Sanchez-Lengeling, J. Wei, B. Lee, E. Reif, P. Wang, W. W. Qian, K. McCloskey, L. Colwell, and A. Wiltschko, “Evaluating attribution for graph neural networks,” Advances in Neural Information Processing Systems , vol. 33, 2020
2020
Closest in time.
A. Tolmachev, A. Sakai, M. Todoriki, and K. Maruhashi, “Bermuda triangles: Gnns fail to detect simple topological structures,” in ICLR 2021 Workshop on Geometrical and Topological Representation Learning , 2021
2021
Closest in time.
L. Cai, J. Li, J. Wang, and S. Ji, “Line graph neural networks for link prediction,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021
2021
Closest in time.
H. Gao, Y. Liu, and S. Ji, “Topology-aware graph pooling networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021
2021
Closest in time.
H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji, “On explainability of graph neural networks via subgraph explorations,” in Proceedings of The 38th International Conference on Machine Learning , 2021, pp. 12 241–12 252
2021
Closest in time.
T. Funke, M. Khosla, and A. Anand, “Hard masking for explaining graph neural networks,” 2021. [Online]. Available: https://openreview.net/forum?id=uDN8pRAdsoC
2021
Closest in time.
M. S. Schlichtkrull, N. D. Cao, and I. Titov, “Interpreting graph neural networks for {nlp} with differentiable edge masking,” in International Conference on Learning Representations , 2021
2021
Closest in time.
X. Wang, Y. Wu, A. Zhang, X. He, and T. seng Chua, “Causal screening to interpret graph neural networks,” 2021. [Online]. Available: https://openreview.net/forum?id=nzKv5vxZfge
2021
Closest in time.
T. Schnake, O. Eberle, J. Lederer, S. Nakajima, K. T. Schutt, K.-R. Muller, and G. Montavon, “Higher-order explanations of graph neural networks via relevant walks.” IEEE transactions on pattern analysis and machine intelligence , vol. PP, 2021
2021
Closest in time.
M. Todoriki, M. Shingu, S. Yano, A. Tolmachev, T. Komikado, and K. Maruhashi, “Semi-automatic reliable explanations for prediction in graphs,” in 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC) . IEEE, 2021, pp. 0311–0320
2021
Closest in time.
L. Holdijk, M. Boon, S. Henckens, and L. de Jong, “[re] parameterized explainer for graph neural network,” in ML Reproducibility Challenge 2020 , 2021. [Online]. Available: https://openreview.net/forum?id=8JHrucviUf
2021
Closest in time.
M. Liu, Y. Luo, L. Wang, Y. Xie, H. Yuan, S. Gui, H. Yu, Z. Xu, J. Zhang, Y. Liu, K. Yan, H. Liu, C. Fu, B. Oztekin, X. Zhang, and S. Ji, “DIG: A turnkey library for diving into graph deep learning research,” arXiv preprint arXiv:2103.12608 , 2021
Original
2021
Closest in time.
Y. Xie, S. Katariya, X. Tang, E. Huang, N. Rao, K. Subbian, and S. Ji, “Task-agnostic graph explanations,” arXiv preprint arXiv:2202.08335 , 2022
Original
2022
Closest in time.
H. Gao and S. Ji, “Graph U-Nets,” in Proceedings of the 36th International Conference on Machine Learning , 2019, pp. 2083–2092
2092
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…