2021

An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networks

Xu, Cong, Li, Xiang, Yang, Min

Understand

Neural networks are susceptible to artificially designed adversarial perturbations.

Recent efforts have shown that imposing certain modifications on classification layer can improve the robustness of the neural networks.
In this paper, we explicitly construct a dense orthogonal weight matrix whose entries have the same magnitude, thereby leading to a novel robust classifier.
The proposed classifier avoids the undesired structural redundancy issue in previous work.

Built on

LeCun Y., Bottou L., Bengio Y. & Haffner P. Gradient-based learning applied to document recognition. In Proceedings of the IEEE
1998
Earlier work this paper cites.
Van der Maaten L. & Hinton G. E. Visualizing data using t-SNE. Journal of Machine Learning Research
2008
Earlier work this paper cites.
Krizhevsky A. & Hinton G. E. Learning multiple layers of features from tiny images. Techinical report, Citeseer
2009
Earlier work this paper cites.
Nair V. & Hinton G. E. Rectified linear units improve restricted boltzmann machines. In International Conference on Machine Learning (ICML)
2010
Earlier work this paper cites.
Szegedy C., Zaremba W., Sutskever I., Bruna J., Erhan D. Goodfellow I. & Fergus R. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR)
2014
Earlier work this paper cites.
Goodfellow I., Shlens J. & Szegedy C. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR)
2015
Earlier work this paper cites.
He K., Zhang X., Ren S. & Jian S. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In International Conference on Computer Vision (ICCV)
2015
Earlier work this paper cites.
He K., Zhang X., Ren S. & Jian S. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2016
Earlier work this paper cites.
Moosavi-Dezfooli S., Fawzi A. & Frossard P. DeepFool: a simple and accurate method to fool deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2016
Earlier work this paper cites.
Papernot N., Mcdaniel P., Wu X., Jha S. & Swami A. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE European Symposium on Security and Privacy
2016
Earlier work this paper cites.
Carlini N. & Wagner D. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (SP)
2017
Earlier work this paper cites.
Rauber J., Brendel W. & Bethge M. Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131
Original
2017
Earlier work this paper cites.
Xiao H., Rasul K. & Vollgraf R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747
Original
2017
Earlier work this paper cites.
Xie D., Xiong J. & Pu S All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2017
Earlier work this paper cites.

Similar

Athalye A., Carlini N. & Wagner D. Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML)
2018
Cited alongside, same era.
Bansal N., Chen X. & Wang Z. Can we gain more from orthogonality regularizations in training deep CNNs? In Advances in Neural Information Processing Systems (NIPS)
2018
Cited alongside, same era.
Huang L., Liu X., Lang B., Yu A. & Li B. Orthogonal weight normalization: solution to optimization over multiple dependent stiefel manifolds in deep neural networks. In Conference on Artificial Intelligence (AAAI)
2018
Cited alongside, same era.
Kannan H., Kurakin A. & Goodfellow I. Adversarial logit pairing. arXiv preprint arXiv:1803.06373
Original
2018
Cited alongside, same era.
Madry A., Makelov A., Schmidt L. & Tsipras D. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR)
2018
Cited alongside, same era.
Pang T., Du C. & Zhu J. Max-mahalanobis linear discriminant analysis networks. In International Conference on Machine Learning (ICML)
2018
Cited alongside, same era.
Su D., Zhang H., Chen H., Yi J., Chen P. & Gao Y. Is robustness the cost of accuracy?-a comprehensive study on the robustness of 18 deep image classification models. In Proceedings of the European Conference on Computer Vision (ECCV)
2018
Cited alongside, same era.
Wan W., Zhong Y., Li T. & Chen J. Rethinking feature distribution for loss functions in image classification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2018
Cited alongside, same era.
Xu W., Evans D. & Qi Y. Feature squeezing: Detecting adversarial examples in deep neural networks. In Network and Distributed Systems Security Symposium (NDSS)
2018
Cited alongside, same era.
Carlini N., Athalye A., Papernot N., Brendel W., Rauber J., Tsipras D., Goodfellow I., Madry A. & Kurakin A. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705
Original
2019
Cited alongside, same era.
Cohen J., Rosenfeld E & Kolter J. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning (ICML)
2019
Cited alongside, same era.
Guo F., Zhao Q., Li X., Kuang X., Zhang J., Han Y. & Tan Y. Detecting adversarial examples via prediction difference for deep neural networks. Information Sciences
2019
Cited alongside, same era.

Then

Mustafa A., Khan S., Hayat M., Goecke R., Shen J. & Shao L. Adversarial defense by restricting the hidden space of deep neural networks. In International Conference on Computer Vision (ICCV)
2019
Later among the works it cites.
Naveiro R., Redondo A., Ríos Insua D., Ruggeri F. Adversarial classification: an adversarial risk analysis approach. International Journal of Approximate Reasoning
2019
Later among the works it cites.
Shafahi A., Najibi M., Ghiasi A., Xu Z., Dickerson J., Studer C., Davis L., Taylor G. & Goldstein T. Adversarial training for free. In Advances in Neural Information Processing Systems (NIPS)
2019
Later among the works it cites.
Tramèr F. & Boneh D. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems (NIPS)
2019
Later among the works it cites.
Wang Jia., Chen Y., Chakraborty R. and Yu S. Orthogonal convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2019
Later among the works it cites.
Zhang D., Zhang T., Lu Y., Zhu Z. & Dong B. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems (NIPS)
2019
Later among the works it cites.
Zhang H., Yu Y., Jiao J., Xing E., Ghaoui L. & Jordan M. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML)
2019
Later among the works it cites.
Croce F. & Hein M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning (ICML)
2020
Later among the works it cites.
Croce F., Andriushchenko M., Sehwag V., Flammarion N., Chiang M., Mittal P. & Hein M. RobustBench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670
Original
2020
Later among the works it cites.
Pang T., Xu K., Dong Y., Du C., Chen N. & Zhu J. Rethinking softmax cross-entropy loss for adversarial robustness. In International Conference On Learning Representations (ICLR)
2020
Later among the works it cites.
Wang Y., Zou D., Yi J., Bailey J., Ma X. & Gu Q. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations (ICLR)
2020
Later among the works it cites.
Xu J., Liu H., Wu D., Zhou F., Gao C. & Jiang L. Generating universal adversarial perturbation with ResNet. Information Sciences
2020
Later among the works it cites.
Xie C., Tan M., Gong B., Wang J., Yuille A. and & Le Q. V. Adversarial examples improve image recognition. In IEEE Conference on Computer Vision and Pattern Recognition
2020
Later among the works it cites.
Wang L., Chen X., Tang R., Yue Y., Zhu Y., Zeng X. & Wang W. Improving adversarial robustness of deep neural networks by using semantic information. Knowledge-Based Systems
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…