Fetching the paper…

Reading the bibliography…

2020

Information-Theoretic Visual Explanation for Black-Box Classifiers

Yi, Jihun, Kim, Eunji, Kim, Siwon et al.

Original
PDF
Post on X

Understand

In this work, we attempt to explain the prediction of any black-box classifier from an information-theoretic perspective.

For each input feature, we compare the classifier outputs with and without that feature using two information-theoretic metrics.
Accordingly, we obtain two attribution maps--an information gain (IG) map and a point-wise mutual information (PMI) map.
IG map provides a class-independent answer to "How informative is each pixel?", and PMI map offers a class-specific explanation of "How much does each pixel support a specific class?" Compared to existing methods, our method improves the correctness of the attribution maps in terms of a quantitative metric.

Information-Theoretic Visual Explanation for Black-Box Classifiers · Around

Built on

K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034 , 2013
Original
2013
Earlier work this paper cites.
A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in Proc. icml , vol. 30, no. 1, 2013, p. 3
2013
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Earlier work this paper cites.
M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in European conference on computer vision . Springer, 2014, pp. 818–833
2014
Earlier work this paper cites.
J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, “Striving for simplicity: The all convolutional net,” arXiv preprint arXiv:1412.6806 , 2014
Original
2014
Earlier work this paper cites.
S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,” PloS one , vol. 10, no. 7, p. e0130140, 2015
2015
Earlier work this paper cites.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 1–9
2015
Earlier work this paper cites.
D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” arXiv preprint arXiv:1610.02136 , 2016
Original
2016
Earlier work this paper cites.
M. T. Ribeiro, S. Singh, and C. Guestrin, “Why should i trust you?: Explaining the predictions of any classifier,” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . ACM, 2016, pp. 1135–1144
2016
Earlier work this paper cites.
L. S. Shapley, 17. A value for n-person games . Princeton University Press, 2016
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al. , “Tensorflow: A system for large-scale machine learning,” arXiv preprint arXiv:1603.04467 , 2016
Original
2016
Earlier work this paper cites.

Similar

A. Shrikumar, P. Greenside, and A. Kundaje, “Learning important features through propagating activation differences,” in Proceedings of the 34th International Conference on Machine Learning-Volume 70 . JMLR. org, 2017, pp. 3145–3153
2017
Cited alongside, same era.
M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in Proceedings of the 34th International Conference on Machine Learning-Volume 70 . JMLR. org, 2017, pp. 3319–3328
2017
Cited alongside, same era.
L. M. Zintgraf, T. S. Cohen, T. Adel, and M. Welling, “Visualizing deep neural network decisions: Prediction difference analysis,” arXiv preprint arXiv:1702.04595 , 2017
Original
2017
Cited alongside, same era.
S. Liang, Y. Li, and R. Srikant, “Enhancing the reliability of out-of-distribution image detection in neural networks,” arXiv preprint arXiv:1706.02690 , 2017
Original
2017
Cited alongside, same era.

Then

J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 7132–7141
2018
Later among the works it cites.
V. Petsiuk, A. Das, and K. Saenko, “Rise: Randomized input sampling for explanation of black-box models,” arXiv preprint arXiv:1806.07421 , 2018
Original
2018
Later among the works it cites.
C.-H. Chang, E. Creager, A. Goldenberg, and D. Duvenaud, “Explaining image classifiers by counterfactual generation,” 2018
2018
Later among the works it cites.
J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang, “Generative image inpainting with contextual attention,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 5505–5514
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg, “Smoothgrad: removing noise by adding noise,” arXiv preprint arXiv:1706.03825 , 2017

Original

2017

Cited alongside, same era.

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 618–626

2017

Cited alongside, same era.

P. Dabkowski and Y. Gal, “Real time image saliency for black box classifiers,” in Advances in Neural Information Processing Systems , 2017, pp. 6967–6976

2017

Cited alongside, same era.

R. C. Fong and A. Vedaldi, “Interpretable explanations of black boxes by meaningful perturbation,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 3429–3437

2017

Cited alongside, same era.

S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in NIPS , 2017, pp. 4768–4777

2017

Cited alongside, same era.

M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” in Advances in Neural Information Processing Systems , 2017, pp. 6626–6637

2017

Cited alongside, same era.

G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 4700–4708

2017

Cited alongside, same era.

I. E. Kumar, S. Venkatasubramanian, C. Scheidegger, and S. Friedler, “Problems with shapley-value-based explanations as feature importance measures,” in International Conference on Machine Learning . PMLR, 2020, pp. 5491–5500

2017

Cited alongside, same era.

J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” in Advances in Neural Information Processing Systems , 2018, pp. 9505–9515

2018

Later among the works it cites.

W. Nie, Y. Zhang, and A. Patel, “A theoretical explanation for perplexing behaviors of backpropagation-based visualizations,” arXiv preprint arXiv:1805.07039 , 2018

2018

Later among the works it cites.

R. Fong, M. Patrick, and A. Vedaldi, “Understanding deep networks via extremal perturbations and smooth masks,” in Proceedings of the IEEE International Conference on Computer Vision , 2019, pp. 2950–2958

2019

Later among the works it cites.

A. Ghorbani, A. Abid, and J. Zou, “Interpretation of neural networks is fragile,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 33, 2019, pp. 3681–3688

2019

Later among the works it cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019, pp. 4171–4186

2019

Later among the works it cites.

U. Hwang, D. Jung, and S. Yoon, “Hexagan: Generative adversarial nets for real world classification,” in International Conference on Machine Learning , 2019, pp. 2921–2930

2019

Later among the works it cites.

S. Kim, J. Yi, E. Kim, and S. Yoon, “Interpretation of nlp models through input marginalization,” arXiv preprint arXiv:2010.13984 , 2020

2020

Closest in time.

C. Frye, D. de Mijolla, T. Begley, L. Cowton, M. Stanley, and I. Feige, “Shapley explainability on the data manifold,” in ICLR , 2021

2021

Closest in time.