2020

Defending against Backdoor Attack on Deep Neural Networks

Cheng, Hao, Xu, Kaidi, Liu, Sijia et al.

Understand

Although deep neural networks (DNNs) have achieved a great success in various computer vision tasks, it is recently found that they are vulnerable to adversarial attacks.

In this paper, we focus on the so-called \textit{backdoor attack}, which injects a backdoor trigger to a small portion of training data (also known as data poisoning) such that the trained DNN induces misclassification while facing examples with this trigger.
To be specific, we carefully study the effect of both real and synthetic backdoor attacks on the internal response of vanilla and backdoored DNNs through the lens of Gard-CAM.
Moreover, we show that the backdoor attack induces a significant bias in neuron activation in terms of the $\ell_\infty$ norm of an activation map compared to its $\ell_1$ and $\ell_2$ norm.

Built on

Intriguing properties of neural networks. In International Conference on Learning Representations
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014 · 2014
Earlier work this paper cites.
Explaining and harnessing adversarial examples. In International Conference on Learning Representations
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015 · 2015
Earlier work this paper cites.
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition . 427–436
Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015 · 2015
Earlier work this paper cites.
Is feature selection secure against training data poisoning?. In International Conference on Machine Learning . 1689–1698
Huang Xiao, Battista Biggio, Gavin Brown, Giorgio Fumera, Claudia Eckert, and Fabio Roli. 2015 · 2015
Earlier work this paper cites.
Learning Deep Features for Discriminative Localization
B. Zhou, A. Khosla, Lapedriza. A., A. Oliva, and A. Torralba. 2016 · 2016
Earlier work this paper cites.
Circnn: accelerating and compressing deep neural networks using block-circulant weight matrices. In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture . 395–408
Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, et al · 2017
Earlier work this paper cites.

Similar

Badnets: Identifying vulnerabilities in the machine learning model supply chain
Original
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017 · 2017
Cited alongside, same era.
Towards poisoning of deep learning algorithms with back-gradient optimization. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security . ACM, 27–38
Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C Lupu, and Fabio Roli. 2017 · 2017
Cited alongside, same era.
Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision . 618–626
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017 · 2017
Cited alongside, same era.
Detecting backdoor attacks on deep neural networks by activation clustering
Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. 2018 · 2018
Cited alongside, same era.
Poison frogs! targeted clean-label poisoning attacks on neural networks. In Advances in Neural Information Processing Systems . 6103–6113
Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018 · 2018
Cited alongside, same era.
Pairwise based deep ranking hashing for histopathology image classification and retrieval
Xiaoshuang Shi, Manish Sapkota, Fuyong Xing, Fujun Liu, Lei Cui, and Lin Yang. 2018 · 2018
Cited alongside, same era.
Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses . Springer, 273–294
Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018a
Cited in the paper.
Trojaning attack on neural networks. In NDSS
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangy Zhang. 2018b
Cited in the paper.
Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective. In International Joint Conference on Artificial Intelligence (IJCAI)
Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. 2019a
Cited in the paper.
Structured Adversarial Attack: Towards General Implementation and Better Interpretability. In International Conference on Learning Representations
Kaidi Xu, Sijia Liu, Pu Zhao, Pin-Yu Chen, Huan Zhang, Quanfu Fan, Deniz Erdogmus, Yanzhi Wang, and Xue Lin. 2019c
Cited in the paper.
On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method. In Proceedings of the IEEE International Conference on Computer Vision . 121–130
Pu Zhao, Sijia Liu, Pin-Yu Chen, Nghia Hoang, Kaidi Xu, Bhavya Kailkhura, and Xue Lin. 2019a
Cited in the paper.
Admm attack: an enhanced adversarial attack for deep neural networks with undetectable distortions. In Proceedings of the 24th Asia and South Pacific Design Automation Conference . 499–505
Pu Zhao, Kaidi Xu, Sijia Liu, Yanzhi Wang, and Xue Lin. 2019b
Cited in the paper.

Then

Spectral signatures in backdoor attacks. In Advances in Neural Information Processing Systems . 8000–8010
Brandon Tran, Jerry Li, and Aleksander Madry. 2018 · 2018
Later among the works it cites.
REQ-YOLO: A resource-aware, efficient quantization framework for object detection on FPGAs. In Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . 33–42
Caiwen Ding, Shuo Wang, Ning Liu, Kaidi Xu, Yanzhi Wang, and Yun Liang. 2019 · 2019
Later among the works it cites.
Hsim-dnn: Hardware simulator for computation-, storage-and power-efficient deep neural networks. In Proceedings of the 2019 on Great Lakes Symposium on VLSI . 81–86
Mengshu Sun, Pu Zhao, Yanzhi Wang, Naehyuck Chang, and Xue Lin. 2019 · 2019
Later among the works it cites.
Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In Advances in Neural Information Processing Systems . IEEE
Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019 · 2019
Later among the works it cites.
Interpreting adversarial examples by activation promotion and suppression
Original
Kaidi Xu, Sijia Liu, Gaoyuan Zhang, Mengshu Sun, Pu Zhao, Quanfu Fan, Chuang Gan, and Xue Lin. 2019b · 2019
Later among the works it cites.
Adversarial robustness vs. model compression, or both. In The IEEE International Conference on Computer Vision (ICCV) , Vol. 2
Shaokai Ye, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin. 2019 · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…