Fetching the paper…

Reading the bibliography…

Improving Model Robustness with Latent Distribution Locally and Globally · Around

2021

Improving Model Robustness with Latent Distribution Locally and Globally

Qian, Zhuang, Zhang, Shufei, Huang, Kaizhu et al.

Original
PDF
Post on X

Understand

In this work, we consider model robustness of deep neural networks against adversarial attacks from a global manifold perspective.

Leveraging both the local and global latent information, we propose a novel adversarial training method through robust optimization, and a tractable way to generate Latent Manifold Adversarial Examples (LMAEs) via an adversarial game between a discriminator and a classifier.
The proposed adversarial training with latent distribution (ATLD) method defends against adversarial attacks by crafting LMAEs with the latent manifold in an unsupervised manner.
ATLD preserves the local and global information of latent manifold and promises improved robustness against adversarial attacks.

Built on

F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget: Continual prediction with lstm.” IET, 1999
1999
Earlier work this paper cites.
X. Nguyen, M. J. Wainwright, and M. I. Jordan, “Estimating divergence functionals and the likelihood ratio by convex risk minimization,” in IEEE Transactions on Information Theory , vol. 56, no. 11. IEEE, 2010, pp. 5847–5861
2010
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in arXiv:1412.6572 , 2014
Original
2014
Earlier work this paper cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” in nature , vol. 521, no. 7553. Nature Publishing Group, 2015, p. 436
2015
Earlier work this paper cites.
C. Lyu, K. Huang, and H.-N. Liang, “A unified gradient regularization family for adversarial examples,” in 2015 IEEE International Conference on Data Mining . IEEE, 2015, pp. 301–309
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
S. Nowozin, B. Cseke, and R. Tomioka, “f-gan: Training generative neural samplers using variational divergence minimization,” in Advances in neural information processing systems , 2016, pp. 271–279
2016
Earlier work this paper cites.
T. Che, Y. Li, A. P. Jacob, Y. Bengio, and W. Li, “Mode regularized generative adversarial networks,” 2016
2016
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis, “Wide residual networks,” in arXiv preprint arXiv:1605.07146 , 2016
Original
2016
Earlier work this paper cites.
V. Fischer, M. C. Kumar, J. H. Metzen, and T. Brox, “Adversarial examples for semantic image segmentation,” in arXiv preprint arXiv:1703.01101 , 2017
Original
2017
Earlier work this paper cites.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in arXiv:1706.06083 , 2017
Original
2017
Earlier work this paper cites.
Y. Song, T. Kim, S. Nowozin, S. Ermon, and N. Kushman, “Pixeldefend: Leveraging generative models to understand and defend against adversarial examples,” in arXiv preprint arXiv:1710.10766 , 2017
Original
2017
Earlier work this paper cites.
T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: a regularization method for supervised and semi-supervised learning,” in arXiv:1704.03976 , 2017
Original
2017
Earlier work this paper cites.
A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” in International conference on machine learning , 2017, pp. 2642–2651
2017
Earlier work this paper cites.
D. Meng and H. Chen, “Magnet: a two-pronged defense against adversarial examples,” in Proceedings of the 2017 ACM SIGSAC conference on computer and communications security , 2017, pp. 135–147
2017
Earlier work this paper cites.
J. H. Metzen, T. Genewein, V. Fischer, and B. Bischoff, “On detecting adversarial perturbations,” in arXiv preprint arXiv:1702.04267 , 2017
Original
2017
Earlier work this paper cites.
Z. Gong, W. Wang, and W.-S. Ku, “Adversarial and clean data are not twins,” in arXiv preprint arXiv:1704.04960 , 2017
Original
2017
Earlier work this paper cites.
K. Grosse, P. Manoharan, N. Papernot, M. Backes, and P. McDaniel, “On the (statistical) detection of adversarial examples,” in arXiv preprint arXiv:1702.06280 , 2017
Original
2017
Earlier work this paper cites.

Similar

M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” in arXiv preprint arXiv:1701.07875 , 2017
Original
2017
Cited alongside, same era.
M. Arjovsky and L. Bottou, “Towards principled methods for training generative adversarial networks,” in Stat , vol. 1050, 2017
2017
Cited alongside, same era.
N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in 2017 IEEE Symposium on Security and Privacy (SP) . IEEE, 2017, pp. 39–57
2017
Cited alongside, same era.
Y. Carmon, A. Raghunathan, L. Schmidt, J. C. Duchi, and P. S. Liang, “Unlabeled data improves adversarial robustness,” in Advances in Neural Information Processing Systems , 2019, pp. 11 192–11 203
2017
Cited alongside, same era.
N. Carlini and D. Wagner, “Audio adversarial examples: Targeted attacks on speech-to-text,” in 2018 IEEE Security and Privacy Workshops (SPW) . IEEE, 2018, pp. 1–7

Then

A. Liu, X. Liu, C. Zhang, H. Yu, Q. Liu, and J. He, “Training robust deep neural networks via adversarial noise propagation,” in arXiv:1909.09034 , 2019
Original
2019
Later among the works it cites.
R. Zhai, T. Cai, D. He, C. Dan, K. He, J. Hopcroft, and L. Wang, “Adversarially robust generalization just requires more unlabeled data,” in arXiv preprint arXiv:1906.00555 , 2019
Original
2019
Later among the works it cites.
C. Mao, Z. Zhong, J. Yang, C. Vondrick, and B. Ray, “Metric learning for adversarial robustness,” in Advances in Neural Information Processing Systems , 2019, pp. 478–489
2019
Later among the works it cites.
A. Sinha, M. Singh, N. Kumari, B. Krishnamurthy, H. Machiraju, and V. Balasubramanian, “Harnessing the vulnerability of latent layers in adversarially trained models,” in arXiv:1905.05186 , 2019
Original
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2018

Cited alongside, same era.

H. Kannan, A. Kurakin, and I. Goodfellow, “Adversarial logit pairing,” in arXiv:1803.06373 , 2018

2018

Cited alongside, same era.

L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, and A. Madry, “Adversarially robust generalization requires more data,” in Advances in Neural Information Processing Systems , 2018, pp. 5014–5026

2018

Cited alongside, same era.

D. Jakubovitz and R. Giryes, “Improving dnn robustness to adversarial attacks using jacobian regularization,” in European Conference on Computer Vision , 2018, pp. 514–529

2018

Cited alongside, same era.

M. Long, Z. Cao, J. Wang, and M. I. Jordan, “Conditional adversarial domain adaptation,” in Advances in Neural Information Processing Systems , 2018, pp. 1640–1650

2018

Cited alongside, same era.

S. Baluja and I. Fischer, “Learning to attack: Adversarial transformation networks,” in Thirty-second aaai conference on artificial intelligence , 2018

2018

Cited alongside, same era.

T. Ma, “Generalization and equilibrium in generative adversarial nets (gans) (invited talk),” in the 50th Annual ACM SIGACT Symposium , 2018

2018

Cited alongside, same era.

Z. You, J. Ye, K. Li, Z. Xu, and P. Wang, “Adversarial noise layer: Regularize neural network by adding noise,” in 2019 IEEE International Conference on Image Processing (ICIP) . IEEE, 2019, pp. 909–913

2019

Cited alongside, same era.

H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan, “Theoretically principled trade-off between robustness and accuracy,” arXiv preprint arXiv:1901.08573 , 2019

2019

Later among the works it cites.

D. Hendrycks, K. Lee, and M. Mazeika, “Using pre-training can improve model robustness and uncertainty,” Proceedings of the International Conference on Machine Learning , 2019

2019

Later among the works it cites.

P. Yang, J. Chen, C.-J. Hsieh, J.-L. Wang, and M. I. Jordan, “Greedy attack and gumbel attack: Generating adversarial examples for discrete data.” Journal of Machine Learning Research , vol. 21, no. 43, pp. 1–36, 2020

2020

Later among the works it cites.

F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” in ICML , 2020

2020

Later among the works it cites.

J. Chen and Q. Gu, “Rays: A ray searching method for hard-label adversarial attack,” in Proceedings of the 26rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2020

2020

Later among the works it cites.

A. Chan, Y. Tay, Y.-S. Ong, and J. Fu, “Jacobian adversarially regularized networks for robustness,” in International Conference on Learning Representation , 2020

2020

Later among the works it cites.

S. Gowal, C. Qin, J. Uesato, T. Mann, and P. Kohli, “Uncovering the limits of adversarial training against norm-bounded adversarial examples,” arXiv preprint arXiv:2010.03593 , 2020

2020

Later among the works it cites.

L. Rice, E. Wong, and J. Z. Kolter, “Overfitting in adversarially robust deep learning,” arXiv preprint arXiv:2002.11569 , 2020

2020

Later among the works it cites.

B. Wu, J. Chen, D. Cai, X. He, and Q. Gu, “Does network width really help adversarial robustness?” arXiv preprint arXiv:2010.01279 , 2020

2020

Later among the works it cites.

S. Zhang, Z. Qian, K. Huang, Q. Wang, R. Zhang, and X. Yi, “Towards better robust generalization with shift consistency regularization,” in ICML , 2021

2021

Closest in time.

S. Zhang, K. Huang, J. Zhu, and Y. Liu, “Manifold adversarial training for supervised and semi-supervised learning,” Neural Networks , vol. 140, pp. 282–293, 2021. [Online]. Available: https://doi.org/10.1016/j.neunet.2021.03.031

2021

Closest in time.

P. Zhai and S. Zhang, “Adversarial information bottleneck,” IEEE Transactions on Neural Networks and Learning Systems , 2022

2022

Closest in time.

X. Yang, Y. Guo, M. Dong, and J.-H. Xue, “Toward certified robustness of distance metric learning,” IEEE Transactions on Neural Networks and Learning Systems , 2022

2022

Closest in time.