2020

RAB: Provable Robustness Against Backdoor Attacks

Weber, Maurice, Xu, Xiaojun, Karlaš, Bojan et al.

Understand

Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial attacks, including evasion and backdoor (poisoning) attacks.

On the defense side, there have been intensive efforts on improving both empirical and provable robustness against evasion attacks; however, the provable robustness against backdoor attacks still remains largely unexplored.
In this paper, we focus on certifying the machine learning model robustness against general threat models, especially backdoor attacks.
We first provide a unified framework via randomized smoothing techniques and show how it can be instantiated to certify the robustness against both evasion and backdoor attacks.

Built on

J. Neyman and E. Pearson, “On the problem of the most efficient tests of statistical hypotheses. 231,” Phil. Trans. Roy. Statistical Soc. A , vol. 289, 1933
1933
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255
2009
Earlier work this paper cites.
A. Krizhevsky, “Learning multiple layers of features from tiny images,” Tech. Rep., 2009
2009
Earlier work this paper cites.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research , vol. 12, pp. 2825–2830, 2011
2011
Earlier work this paper cites.
B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks against support vector machines,” in Proceedings of the 29th International Coference on Machine Learning , 2012, p. 1467–1474
2012
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” International Conference on Learning Representations , 2015
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
B. Li, Y. Wang, A. Singh, and Y. Vorobeychik, “Data poisoning attacks on factorization-based collaborative filtering,” in Advances in neural information processing systems , 2016, pp. 1885–1893
2016
Earlier work this paper cites.
X. Cao and N. Z. Gong, “Mitigating evasion attacks to deep neural networks via region-based classification,” in Proceedings of the 33rd Annual Computer Security Applications Conference , 2017, pp. 278–287
2017
Earlier work this paper cites.
N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods,” in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security . ACM, 2017, pp. 3–14
2017
Earlier work this paper cites.
X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Targeted backdoor attacks on deep learning systems using data poisoning,” arXiv:1712.05526 , 2017
Original
2017
Earlier work this paper cites.
D. Dua and C. Graff, “UCI machine learning repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml
2017
Earlier work this paper cites.
T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnerabilities in the machine learning model supply chain,” arXiv:1708.06733 , 2017
Original
2017
Earlier work this paper cites.
L. Muñoz-González, B. Biggio, A. Demontis, A. Paudice, V. Wongrassamee, E. C. Lupu, and F. Roli, “Towards poisoning of deep learning algorithms with back-gradient optimization,” in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security . ACM, 2017, pp. 27–38
2017
Earlier work this paper cites.
J. Steinhardt, P. W. Koh, and P. Liang, “Certified defenses for data poisoning attacks,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , 2017, pp. 3520–3532
2017
Earlier work this paper cites.
C. Yang, Q. Wu, H. Li, and Y. Chen, “Generative poisoning attack method against neural networks,” arXiv:1703.01340 , 2017
Original
2017
Earlier work this paper cites.
A. Athalye, N. Carlini, and D. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” in International conference on machine learning . PMLR, 2018, pp. 274–283
2018
Earlier work this paper cites.
X. Liu, M. Cheng, H. Zhang, and C.-J. Hsieh, “Towards robust neural networks via random self-ensemble,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 369–385
2018
Earlier work this paper cites.
X. Ma, B. Li, Y. Wang, S. M. Erfani, S. Wijewickrema, G. Schoenebeck, M. E. Houle, D. Song, and J. Bailey, “Characterizing adversarial subspaces using local intrinsic dimensionality,” in International Conference on Learning Representations , 2018
2018
Earlier work this paper cites.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in International Conference on Learning Representations , 2018
2018
Earlier work this paper cites.

Similar

M.-I. Nicolae, M. Sinn, M. N. Tran, B. Buesser, A. Rawat, M. Wistuba, V. Zantedeschi, N. Baracaldo, B. Chen, H. Ludwig et al. , “Adversarial robustness toolbox v1. 0.0,” arXiv:1807.01069 , 2018
Original
2018
Cited alongside, same era.
A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer, T. Dumitras, and T. Goldstein, “Poison frogs! targeted clean-label poisoning attacks on neural networks,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems , 2018, p. 6106–6116
2018
Cited alongside, same era.
B. Tran, J. Li, and A. Madry, “Spectral signatures in backdoor attacks,” in Advances in Neural Information Processing Systems , 2018, pp. 8000–8010
2018
Cited alongside, same era.
S. Wang, X. Wang, S. Ye, P. Zhao, and X. Lin, “Defending dnn adversarial attacks with pruning and logits augmentation,” in 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP) . IEEE, 2018, pp. 1144–1148
2018
Cited alongside, same era.
C. Xiao, B. Li, J.-Y. Zhu, W. He, M. Liu, and D. Song, “Generating adversarial examples with adversarial networks,” in Proceedings of the 27th International Joint Conference on Artificial Intelligence . AAAI Press, 2018, p. 3905–3911
2018
Cited alongside, same era.
W. Xu, D. Evans, and Y. Qi, “Feature squeezing: Detecting adversarial examples in deep neural networks,” in 25th Annual Network and Distributed System Security Symposium . The Internet Society, 2018
2018
Cited alongside, same era.
B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava, “Detecting backdoor attacks on deep neural networks by activation clustering,” in SafeAI@ AAAI , 2019
2019
Cited alongside, same era.
H. Chen, C. Fu, J. Zhao, and F. Koushanfar, “Deepinspect: a black-box trojan detection and mitigation framework for deep neural networks,” in Proceedings of the 28th International Joint Conference on Artificial Intelligence . AAAI Press, 2019, pp. 4658–4664
2019
Cited alongside, same era.
J. Cohen, E. Rosenfeld, and Z. Kolter, “Certified adversarial robustness via randomized smoothing,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, vol. 97, 09–15 Jun 2019, pp. 1310–1320
2019
Cited alongside, same era.
Y. Gao, C. Xu, D. Wang, S. Chen, D. C. Ranasinghe, and S. Nepal, “Strip: A defence against trojan attacks on deep neural networks,” in Proceedings of the 35th Annual Computer Security Applications Conference . New York, NY, USA: Association for Computing Machinery, 2019, p. 113–125
2019
Cited alongside, same era.
T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “Badnets: Evaluating backdooring attacks on deep neural networks,” IEEE Access , vol. 7, pp. 47 230–47 244, 2019
2019
Cited alongside, same era.
M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana, “Certified robustness to adversarial examples with differential privacy,” in 2019 IEEE Symposium on Security and Privacy (SP) , 2019, pp. 656–672
2019
Cited alongside, same era.

Then

A. Saha, A. Subramanya, and H. Pirsiavash, “Hidden trigger backdoor attacks,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, 2020, pp. 11 957–11 965
2020
Closest in time.
M. Sun, S. Agarwal, and J. Z. Kolter, “Poisoned classifiers are not only backdoored, they are fundamentally broken,” in arXiv:2010.09080 , 2020
Original
2020
Closest in time.
B. Wang, X. Cao, N. Z. Gong et al. , “On certifying robustness against backdoor attacks via randomized smoothing,” CVPR 2020 Workshop on Adversarial Machine Learning in Computer Vision , 2020
2020
Closest in time.
Wikipedia contributors, “Sha-2 — Wikipedia, the free encyclopedia,” 2020, [Online; accessed 18-March-2020]. [Online]. Available: https://en.wikipedia.org/w/index.php?title=SHA-2&oldid=944705336
2020
Closest in time.
G. Yang, T. Duan, J. E. Hu, H. Salman, I. Razenshteyn, and J. Li, “Randomized smoothing of all shapes and sizes,” in International Conference on Machine Learning . PMLR, 2020, pp. 10 693–10 705
2020
Closest in time.
Z. Yang, Z. Zhao, H. Pei, B. Wang, B. Karlas, J. Liu, H. Guo, B. Li, and C. Zhang, “End-to-end robustness for sensing-reasoning machine learning pipelines,” arXiv:2003.00120 , 2020
Original
2020
Closest in time.
H. Zhong, C. Liao, A. C. Squicciarini, S. Zhu, and D. Miller, “Backdoor embedding in convolutional neural network models via invisible perturbation,” in Proceedings of the Tenth ACM Conference on Data and Application Security and Privacy , 2020, pp. 97–108
2020
Closest in time.
E. Borgnia, V. Cherepanova, L. Fowl, A. Ghiasi, J. Geiping, M. Goldblum, T. Goldstein, and A. Gupta, “Strong data augmentation sanitizes poisoning and backdoor attacks without an accuracy tradeoff,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 3855–3859
2021
Closest in time.
J. Hayase, W. Kong, R. Somani, and S. Oh, “Spectre: defending against backdoor attacks using robust statistics,” in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18–24 Jul 2021, pp. 4129–4139. [Online]. Available: https://proceedings.mlr.press/v139/hayase21a.html
2021
Closest in time.
J. Jia, X. Cao, and N. Z. Gong, “Intrinsic certified robustness of bagging against data poisoning attacks,” in AAAI , 2021
2021
Closest in time.
A. Levine and S. Feizi, “Deep partition aggregation: Provable defenses against general poisoning attacks,” in 9th International Conference on Learning Representations , 2021
2021
Closest in time.
L. Li, M. Weber, X. Xu, L. Rimanic, B. Kailkhura, T. Xie, C. Zhang, and B. Li, “Tss: Transformation-specific smoothing for robustness certification,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , 2021, p. 535–557
2021
Closest in time.
Y. Ren, L. Li, and J. Zhou, “Simtrojan: Stealthy backdoor attack,” in 2021 IEEE International Conference on Image Processing (ICIP) . IEEE, 2021, pp. 819–823
2021
Closest in time.
J. D. Romano, T. T. Le, W. La Cava, J. T. Gregg, D. J. Goldberg, P. Chakraborty, N. L. Ray, D. Himmelstein, W. Fu, and J. H. Moore, “PMLB v1.0: an open-source dataset collection for benchmarking machine learning methods,” Bioinformatics , vol. 38, no. 3, pp. 878–880, 2021
2021
Closest in time.
A. Schwarzschild, M. Goldblum, A. Gupta, J. P. Dickerson, and T. Goldstein, “Just how toxic is data poisoning? a unified benchmark for backdoor and data poisoning attacks,” in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18–24 Jul 2021, pp. 9389–9398
2021
Closest in time.
——, “Just how toxic is data poisoning? a unified benchmark for backdoor and data poisoning attacks,” in International Conference on Machine Learning . PMLR, 2021, pp. 9389–9398
2021
Closest in time.
D. Tang, X. Wang, H. Tang, and K. Zhang, “Demon in the variant: Statistical analysis of { \{ DNNs } \} for robust backdoor contamination detection,” in 30th USENIX Security Symposium , 2021, pp. 1541–1558
2021
Closest in time.
C. Xie, M. Chen, P.-Y. Chen, and B. Li, “Crfl: Certifiably robust federated learning against backdoor attacks,” in International Conference on Machine Learning . PMLR, 2021, pp. 11 372–11 382
2021
Closest in time.
X. Xu, Q. Wang, H. Li, N. Borisov, C. A. Gunter, and B. Li, “Detecting ai trojans using meta neural analysis,” in 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 2021, pp. 103–120
2021
Closest in time.
J. Jia, Y. Liu, X. Cao, and N. Z. Gong, “Certified robustness of nearest neighbors against data poisoning and backdoor attacks,” in AAAI , 2022
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…