2018

To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression

Zhao, Yiren, Shumailov, Ilia, Mullins, Robert et al.

Understand

As deep neural networks (DNNs) become widely used, pruned and quantised models are becoming ubiquitous on edge devices; such compressed DNNs are popular for lowering computational requirements.

Meanwhile, recent studies show that adversarial samples can be effective at making DNNs misclassify.
We, therefore, investigate the extent to which adversarial samples are transferable between uncompressed and compressed DNNs.
We find that adversarial samples remain transferable for both pruned and quantised models.

Built on

MNIST handwritten digit database
Yann LeCun, Corinna Cortes, and CJ Burges · 2010
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Intriguing properties of neural networks
Original
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus · 2013
Earlier work this paper cites.
The CIFAR-10 dataset
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton · 2014
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun · 2015
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2015
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy · 2015
Earlier work this paper cites.
LeNet-5, convolutional neural networks
Yann LeCun et al · 2015
Earlier work this paper cites.
Bidirectional attention flow for machine comprehension
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi · 2016
Earlier work this paper cites.
EIE: Efficient Inference Engine on Compressed Deep Neural Network
Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz, and William J. Dally · 2016
Earlier work this paper cites.
Hidden Voice Commands
Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou · 2016
Earlier work this paper cites.

Similar

Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks
Yu-Hsin Chen, Joel Emer, and Vivienne Sze · 2016
Cited alongside, same era.
Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding
Song Han, Huizi Mao, and William J Dally · 2016
Cited alongside, same era.
Dynamic network surgery for efficient DNNs
Yiwen Guo, Anbang Yao, and Yurong Chen · 2016
Cited alongside, same era.
Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio · 2016
Cited alongside, same era.
Ternary weight networks
Fengfu Li, Bo Zhang, and Bin Liu · 2016
Cited alongside, same era.
Fixed point quantization of deep convolutional networks
Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy · 2016
Cited alongside, same era.
The limitations of deep learning in adversarial settings
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami · 2016
Cited alongside, same era.
Transferability in machine learning: from phenomena to black-box attacks using adversarial samples
Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow · 2016
Cited alongside, same era.
Adversarial perturbations of deep neural networks
David Warde-Farley and Ian Goodfellow · 2016
Cited alongside, same era.
Adversarial examples in the physical world
Original
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio · 2016
Cited alongside, same era.

Then

DeepFool: a simple and accurate method to fool deep neural networks
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard · 2016
Later among the works it cites.
SCNN: An accelerator for compressed-sparse convolutional neural networks
A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally · 2017
Later among the works it cites.
Quantized neural networks: Training neural networks with low precision weights and activations
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio · 2017
Later among the works it cites.
The space of transferable adversarial examples
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel · 2017
Later among the works it cites.
Practical black-box attacks against machine learning
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami · 2017
Later among the works it cites.
Robust Physical-World Attacks on Deep Learning Visual Classification
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song · 2018
Closest in time.
Zena: Zero-aware neural network accelerator
Dongyoung Kim, Junwhan Ahn, and Sungjoo Yoo · 2018
Closest in time.
Mayo: A framework for auto-generating hardware friendly deep neural networks
Yiren Zhao, Xitong Gao, Robert Mullins, and Chengzhong Xu · 2018
Closest in time.
Rethinking the value of network pruning
Original
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell · 2018
Closest in time.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Original
Jonathan Frankle and Michael Carbin · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…