2020

Towards Backdoor Attacks and Defense in Robust Machine Learning Models

Soremekun, Ezekiel, Udeshi, Sakshi, Chattopadhyay, Sudipta

Understand

The introduction of robust optimisation has pushed the state-of-the-art in defending against adversarial attacks.

Notably, the state-of-the-art projected gradient descent (PGD)-based training method has been shown to be universally and reliably effective in defending against adversarial inputs.
This robustness approach uses PGD as a reliable and universal "first-order adversary".
However, the behaviour of such optimisation has not been studied in the light of a fundamentally different class of attacks called backdoors.

Built on

S. Udeshi, S. Peng, G. Woo, L. Loh, L. Rawshan, and S. Chattopadhyay, “Model agnostic defence against backdoor attacks in machine learning,” CoRR , vol. abs/1908.02203, 2019. [Online]. Available: http://arxiv.org/abs/1908.02203
Original
1908
Earlier work this paper cites.
S. Kaur, J. Cohen, and Z. C. Lipton, “Are perceptually-aligned gradients a general property of robust classifiers?” CoRR , vol. abs/1910.08640, 2019. [Online]. Available: http://arxiv.org/abs/1910.08640
Original
1910
Earlier work this paper cites.
M. Du, R. Jia, and D. Song, “Robust anomaly detection and backdoor attack detection via differential privacy,” CoRR , vol. abs/1911.07116, 2019. [Online]. Available: http://arxiv.org/abs/1911.07116
Original
1911
Earlier work this paper cites.
K. Fukunaga and L. D. Hostetler, “The estimation of the gradient of a density function, with applications in pattern recognition,” IEEE Trans. Information Theory , vol. 21, no. 1, pp. 32–40, 1975
1975
Earlier work this paper cites.
K. C. Gowda and G. Krishna, “Agglomerative clustering using the concept of mutual nearest neighbourhood,” Pattern recognition , vol. 10, no. 2, pp. 105–112, 1978
1978
Earlier work this paper cites.
A. Maćkiewicz and W. Ratajczak, “Principal components analysis (pca),” Computers & Geosciences , vol. 19, no. 3, pp. 303–342, 1993
1993
Earlier work this paper cites.
Y. LeCun, C. Cortes, and C. J. Burges, “The mnist database of handwritten digits, 1998,” URL http://yann. lecun. com/exdb/mnist , vol. 10, p. 34, 1998
1998
Earlier work this paper cites.
A. Ng, M. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” Advances in neural information processing systems , vol. 14, 2001
2001
Earlier work this paper cites.
A. Likas, N. Vlassis, and J. J. Verbeek, “The global k-means clustering algorithm,” Pattern recognition , vol. 36, no. 2, pp. 451–461, 2003
2003
Earlier work this paper cites.
L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of machine learning research , vol. 9, no. Nov, pp. 2579–2605, 2008
2008
Earlier work this paper cites.
D. Dueck, Affinity propagation: clustering data by passing messages . Citeseer, 2009
2009
Earlier work this paper cites.
A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , 2014
2014
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
N. Papernot, P. D. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a defense to adversarial perturbations against deep neural networks,” in IEEE Symposium on Security and Privacy, SP 2016, San Jose, CA, USA, May 22-26, 2016 , 2016, pp. 582–597
2016
Earlier work this paper cites.
N. Papernot, P. D. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami, “The limitations of deep learning in adversarial settings,” in IEEE European Symposium on Security and Privacy, EuroS&P 2016, Saarbrücken, Germany, March 21-24, 2016 , 2016, pp. 372–387
2016
Earlier work this paper cites.
T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnerabilities in the machine learning model supply chain,” CoRR , vol. abs/1708.06733, 2017. [Online]. Available: http://arxiv.org/abs/1708.06733
Original
2017
Earlier work this paper cites.
N. Papernot, P. D. McDaniel, I. J. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, AsiaCCS 2017, Abu Dhabi, United Arab Emirates, April 2-6, 2017 , 2017, pp. 506–519
2017
Earlier work this paper cites.
Z. Yi, H. Zhang, P. Tan, and M. Gong, “Dualgan: Unsupervised dual learning for image-to-image translation,” in The IEEE International Conference on Computer Vision (ICCV) , Oct 2017
2017
Earlier work this paper cites.
J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in The IEEE International Conference on Computer Vision (ICCV) , Oct 2017
2017
Earlier work this paper cites.
P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , July 2017
2017
Earlier work this paper cites.
M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-to-image translation networks,” in Advances in Neural Information Processing Systems 30 , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc., 2017, pp. 700–708. [Online]. Available: http://papers.nips.cc/paper/6672-unsupervised-image-to-image-translation-networks.pdf
2017
Earlier work this paper cites.
L. McInnes, J. Healy, and S. Astels, “hdbscan: Hierarchical density based clustering.” J. Open Source Softw. , vol. 2, no. 11, p. 205, 2017
2017
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf. (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms
2017
Earlier work this paper cites.
A. Sinha, H. Namkoong, and J. Duchi, “Certifying some distributional robustness with principled adversarial training,” arXiv preprint arXiv:1710.10571 , 2017
Original
2017
Earlier work this paper cites.

Similar

M. Abbasi and C. Gagné, “Robustness to adversarial examples through an ensemble of specialists,” arXiv preprint arXiv:1702.06856 , 2017
Original
2017
Cited alongside, same era.
W. He, J. Wei, X. Chen, N. Carlini, and D. Song, “Adversarial example defense: Ensembles of weak defenses are not strong,” in 11th USENIX Workshop on Offensive Technologies, WOOT 2017, Vancouver, BC, Canada, August 14-15, 2017 , 2017
2017
Cited alongside, same era.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings , 2018
2018
Cited alongside, same era.
B. Tran, J. Li, and A. Madry, “Spectral signatures in backdoor attacks,” in Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada , 2018, pp. 8011–8021
2018
Cited alongside, same era.
E. Chou, F. Tramèr, G. Pellegrino, and D. Boneh, “Sentinet: Detecting physical attacks against deep learning systems,” arXiv preprint arXiv:1812.00292 , 2018
Original
2018
Cited alongside, same era.
K. Liu, B. Dolan-Gavitt, and S. Garg, “Fine-pruning: Defending against backdooring attacks on deep neural networks,” in Research in Attacks, Intrusions, and Defenses - 21st International Symposium, RAID 2018, Heraklion, Crete, Greece, September 10-12, 2018, Proceedings , 2018, pp. 273–294
2018
Cited alongside, same era.
Y. Liu, S. Ma, Y. Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang, “Trojaning attack on neural networks,” in 25nd Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-221, 2018 . The Internet Society, 2018
2018
Cited alongside, same era.
E. Wong, F. Schmidt, J. H. Metzen, and J. Z. Kolter, “Scaling provable adversarial defenses,” in Advances in Neural Information Processing Systems , 2018, pp. 8400–8409
2018
Cited alongside, same era.
E. Wong and Z. Kolter, “Provable defenses against adversarial examples via the convex outer adversarial polytope,” in International Conference on Machine Learning , 2018, pp. 5286–5295
2018
Cited alongside, same era.
A. Raghunathan, J. Steinhardt, and P. Liang, “Certified defenses against adversarial examples,” arXiv preprint arXiv:1801.09344 , 2018
Original
2018
Cited alongside, same era.
L. McInnes, J. Healy, N. Saul, and L. Großberger, “Umap: Uniform manifold approximation and projection,” Journal of Open Source Software , vol. 3, no. 29, p. 861, 2018
2018
Cited alongside, same era.
M. Zou, Y. Shi, C. Wang, F. Li, W. Song, and Y. Wang, “Potrojan: powerful neural-level trojan designs in deep learning models,” arXiv preprint arXiv:1802.03043 , 2018
Original
2018
Cited alongside, same era.

Then

D. Tang, X. Wang, H. Tang, and K. Zhang, “Demon in the variant: Statistical analysis of dnns for robust backdoor contamination detection,” arXiv preprint arXiv:1908.00686 , 2019
Original
2019
Later among the works it cites.
X. Xu, Q. Wang, H. Li, N. Borisov, C. A. Gunter, and B. Li, “Detecting ai trojans using meta neural analysis,” arXiv preprint arXiv:1910.03137 , 2019
Original
2019
Later among the works it cites.
D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry, “Robustness may be at odds with accuracy,” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 , 2019
2019
Later among the works it cites.
L. Engstrom, A. Ilyas, S. Santurkar, and D. Tsipras, “Robustness (python library),” 2019. [Online]. Available: https://github.com/MadryLab/robustness
2019
Later among the works it cites.
T. Pang, K. Xu, C. Du, N. Chen, and J. Zhu, “Improving adversarial robustness via promoting ensemble diversity,” in International Conference on Machine Learning . PMLR, 2019, pp. 4970–4979
2019
Later among the works it cites.
S. Kariyappa and M. K. Qureshi, “Improving adversarial robustness of ensembles with diversity training,” arXiv preprint arXiv:1901.09981 , 2019
Original
2019
Later among the works it cites.
A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,” Advances in neural information processing systems , vol. 32, 2019
2019
Later among the works it cites.
S. Hong, V. Chandrasekaran, Y. Kaya, T. Dumitraş, and N. Papernot, “On the effectiveness of mitigating data poisoning attacks with gradient shaping,” arXiv preprint arXiv:2002.11497 , 2020
Original
2020
Closest in time.
Z. Xiang, D. J. Miller, and G. Kesidis, “Detection of backdoors in trained classifiers without access to the training set,” IEEE Transactions on Neural Networks and Learning Systems , 2020
2020
Closest in time.
A. K. Veldanda, K. Liu, B. Tan, P. Krishnamurthy, F. Khorrami, R. Karri, B. Dolan-Gavitt, and S. Garg, “Nnoculation: Broad spectrum and targeted treatment of backdoored dnns,” arXiv preprint arXiv:2002.08313 , 2020
Original
2020
Closest in time.
H. Zhong, C. Liao, A. C. Squicciarini, S. Zhu, and D. Miller, “Backdoor embedding in convolutional neural network models via invisible perturbation,” in Proceedings of the Tenth ACM Conference on Data and Application Security and Privacy , 2020, pp. 97–108
2020
Closest in time.
Y. Meng, J. Su, J. O’Kane, and P. Jamshidi, “Athena: A framework based on diverse weak defenses for building adversarial defense,” arXiv preprint arXiv:2001.00308 , 2020
Original
2020
Closest in time.
F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” in ICML , 2020
2020
Closest in time.
F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein, “Robustbench: a standardized adversarial robustness benchmark,” arXiv preprint arXiv:2010.09670 , 2020
Original
2020
Closest in time.
E. Bagdasaryan and V. Shmatikov, “Blind backdoors in deep learning models,” arXiv preprint arXiv:2005.03823 , 2020
Original
2020
Closest in time.
C. Guo, R. Wu, and K. Q. Weinberger, “Trojannet: Embedding hidden trojan horse models in neural networks,” arXiv preprint arXiv:2002.10078 , 2020
Original
2020
Closest in time.
Y. Li, B. Wu, Y. Jiang, Z. Li, and S.-T. Xia, “Backdoor learning: A survey,” arXiv preprint arXiv:2007.08745 , 2020
Original
2020
Closest in time.
R. Pang, H. Shen, X. Zhang, S. Ji, Y. Vorobeychik, X. Luo, A. Liu, and T. Wang, “A tale of evil twins: Adversarial inputs versus poisoned models,” in Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security , 2020, pp. 85–99
2020
Closest in time.
C.-H. Weng, Y.-T. Lee, and S.-H. B. Wu, “On the trade-off between adversarial and backdoor robustness,” Advances in Neural Information Processing Systems , vol. 33, pp. 11 973–11 983, 2020
2020
Closest in time.
“Microsoft azure cognitive services,” 2021. [Online]. Available: https://azure.microsoft.com/en-us/services/cognitive-services/
2021
Closest in time.
“Google automl,” 2021. [Online]. Available: https://cloud.google.com/automl
2021
Closest in time.
R. Li, H. Zhang, P. Yang, C.-C. Huang, A. Zhou, B. Xue, and L. Zhang, “Ensemble defense with data diversity: Weak correlation implies strong robustness,” arXiv preprint arXiv:2106.02867 , 2021
Original
2021
Closest in time.
——, “Mind the box: l 1 l_{1} -apgd for sparse adversarial attacks on image classifiers,” in ICML , 2021
2021
Closest in time.
Y. Yao, H. Li, H. Zheng, and B. Y. Zhao, “Latent backdoor attacks on deep neural networks,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK, November 11-15, 2019 , 2019, pp. 2041–2055. [Online]. Available: https://doi.org/10.1145/3319535.3354209
2055
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…