2017

Real Time Image Saliency for Black Box Classifiers

Dabkowski, Piotr, Gal, Yarin

Understand

In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier.

We train a masking model to manipulate the scores of the classifier by masking salient parts of the input image.
Our model generalises well to unseen images and requires a single forward pass to perform saliency detection, therefore suitable for use in real-time systems.
We test our approach on CIFAR-10 and ImageNet datasets and show that the produced saliency maps are easily interpretable, sharp, and free of artifacts.

Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Deep inside convolutional networks: Visualising image classification models and saliency maps
Original
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman · 2013
Earlier work this paper cites.
Intriguing properties of neural networks
Original
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus · 2013
Earlier work this paper cites.
Visualizing and Understanding Convolutional Networks
Original
Matthew D. Zeiler and Rob Fergus · 2013
Earlier work this paper cites.
FitNets: Hints for Thin Deep Nets
Original
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Original
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.

Striving for simplicity: The all convolutional net
Original
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller · 2014
Cited alongside, same era.
Going deeper with convolutions
Original
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2014
Cited alongside, same era.
Visualizing and understanding convolutional networks
Matthew D Zeiler and Rob Fergus · 2014
Cited alongside, same era.
Object Detectors Emerge in Deep Scene CNNs
Original
Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba · 2014
Cited alongside, same era.
On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek · 2015
Cited alongside, same era.
Look and think twice: Capturing top-down visual attention with feedback convolutional neural networks
Chunshui Cao, Xianming Liu, Yi Yang, Yinan Yu, Jiang Wang, Zilei Wang, Yongzhen Huang, Liang Wang, Chang Huang, Wei Xu, Deva Ramanan, and Thomas S. Huang · 2015
Cited alongside, same era.
Deep residual learning for image recognition
Original
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.

U-net: Convolutional networks for biomedical image segmentation
Original
Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015
Later among the works it cites.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei · 2015
Later among the works it cites.
Learning deep features for discriminative localization
Original
Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba · 2015
Later among the works it cites.
Why should i trust you?: Explaining the predictions of any classifier
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin · 2016
Later among the works it cites.
Top-down neural attention by excitation backprop
Jianming Zhang, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff · 2016
Later among the works it cites.
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Original
Ruth Fong and Andrea Vedaldi · 2017
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…