2019

Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet

Brendel, Wieland, Bethge, Matthias

Understand

Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions.

We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain.
Our model, a simple variant of the ResNet-50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering.
This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and reaches a surprisingly high accuracy on ImageNet (87.6% top-5 for 33 x 33 px features and Alexnet performance for 17 x 17 px features).

Built on

Visual categorization with bags of keypoints
G. Csurka, C. Bray, C. Dance, and L. Fan · 2004
Earlier work this paper cites.
Creating efficient codebooks for visual recognition
Frédéric Jurie and Bill Triggs · 2005
Earlier work this paper cites.
Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories
Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce · 2006
Earlier work this paper cites.
Local features and kernels for classification of texture and object categories: A comprehensive study
Jianguo Zhang, Marcin Marszalek, Svetlana Lazebnik, and Cordelia Schmid · 2007
Earlier work this paper cites.
How to explain individual classification decisions
David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller · 2010
Earlier work this paper cites.
Introduction to the bag of features paradigm for image classification and retrieval
Original
Stephen O’Hara and Bruce A. Draper · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Return of the devil in the details: Delving deep into convolutional nets
Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman · 2014
Earlier work this paper cites.
Multi-scale orderless pooling of deep convolutional activation features
Yunchao Gong, Liwei Wang, Ruiqi Guo, and Svetlana Lazebnik · 2014
Earlier work this paper cites.
Weakly supervised semantic segmentation with convolutional networks
Original
Pedro H. O. Pinheiro and Ronan Collobert · 2014
Earlier work this paper cites.
Netvlad: CNN architecture for weakly supervised place recognition
Original
Relja Arandjelovic, Petr Gronát, Akihiko Torii, Tomás Pajdla, and Josef Sivic · 2015
Earlier work this paper cites.

Similar

Aggregating deep convolutional features for image retrieval
Original
Artem Babenko and Victor S. Lempitsky · 2015
Cited alongside, same era.
Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks
Original
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge · 2015
Cited alongside, same era.
Deep residual learning for image recognition
Original
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.
Distilling the knowledge in a neural network, 2015
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Cited alongside, same era.
Exploiting local features from deep networks for image retrieval
Joe Yue-Hei Ng, Fan Yang, and Larry S. Davis · 2015
Cited alongside, same era.
The application of two-level attention models in deep convolutional neural network for fine-grained image classification
Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiaxing Zhang, Yuxin Peng, and Zheng Zhang · 2015
Cited alongside, same era.
Learning deep features for discriminative localization
Original
Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba · 2015
Cited alongside, same era.
Scale coding bag of deep features for human attribute and action recognition
Original
Fahad Shahbaz Khan, Joost van de Weijer, Rao Muhammad Anwer, Andrew D. Bagdanov, Michael Felsberg, and Jorma Laaksonen · 2016
Cited alongside, same era.
Bags of local convolutional features for scalable instance search
Eva Mohedano, Kevin McGuinness, Noel E. O’Connor, Amaia Salvador, Ferran Marques, and Xavier Giro-i Nieto · 2016
Cited alongside, same era.
Deep fishernet for object classification
Original
Peng Tang, Xinggang Wang, Baoguang Shi, Xiang Bai, Wenyu Liu, and Zhuowen Tu · 2016
Cited alongside, same era.
Hcp: A flexible cnn framework for multi-label image classification
Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Yao Zhao, and Shuicheng Yan · 2016
Cited alongside, same era.

Then

Towards better understanding of gradient-based attribution methods for deep neural networks, 2017
Original
Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross · 2017
Later among the works it cites.
Local deep descriptors in bag-of-words for image retrieval
Jiewei Cao, Zi Huang, and Heng Tao Shen · 2017
Later among the works it cites.
Bag of visual words model with deep spatial features for geographical scene classification
Jiangfan Feng, Yuanyuan Liu, and Lin Wu · 2017
Later among the works it cites.
Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions
Original
Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin · 2017
Later among the works it cites.
Scaling the scattering transform: Deep hybrid networks
Original
Edouard Oyallon, Eugene Belilovsky, and Sergey Zagoruyko · 2017
Later among the works it cites.
Learning important features through propagating activation differences
Original
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje · 2017
Later among the works it cites.
Axiomatic attribution for deep networks
Original
Mukund Sundararajan, Ankur Taly, and Qiqi Yan · 2017
Later among the works it cites.
Deep patch learning for weakly supervised object classification and discovery
Original
Peng Tang, Xinggang Wang, Zilong Huang, Xiang Bai, and Wenyu Liu · 2017
Later among the works it cites.
Visualizing deep neural network decisions: Prediction difference analysis
Original
Luisa M. Zintgraf, Taco S. Cohen, Tameem Adel, and Max Welling · 2017
Later among the works it cites.
This looks like that: deep learning for interpretable image recognition
Original
Chaofan Chen, Oscar Li, Alina Barnett, Jonathan Su, and Cynthia Rudin · 2018
Later among the works it cites.
Learning how to explain neural networks: Patternnet and patternattribution
Pieter-Jan Kindermans, Kristof T Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, and Sven Dähne · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…