2019

Achieving Robustness in the Wild via Adversarial Mixing with Disentangled Representations

Gowal, Sven, Qin, Chongli, Huang, Po-Sen et al.

Understand

Recent research has made the surprising finding that state-of-the-art deep learning models sometimes fail to generalize to small variations of the input.

Adversarial training has been shown to be an effective approach to overcome this problem.
However, its application has been limited to enforcing invariance to analytically defined transformations like $\ell_p$-norm bounded perturbations.
Such perturbations do not necessarily cover plausible real-world variations that preserve the semantics of the input (such as a change in lighting conditions).

Built on

V. Vapnik, “Statistical learning theory,” 1998
1998
Earlier work this paper cites.
Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010. [Online]. Available:
2010
Earlier work this paper cites.
A. Torralba, A. A. Efros
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in
2012
Earlier work this paper cites.
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and others, “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,”
2012
Earlier work this paper cites.
T. M. Cover and J. A. Thomas,
2012
Earlier work this paper cites.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,”
Original
2013
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,”
Original
2014
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”
Original
2014
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
Original
2014
Earlier work this paper cites.
N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a defense to adversarial perturbations against deep neural networks,”
Original
2015
Earlier work this paper cites.
Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in
2015
Earlier work this paper cites.
I. Goodfellow, Y. Bengio, and A. Courville,
2016
Earlier work this paper cites.
K. D. Julian, J. Lopez, J. S. Brush, M. P. Owen, and M. J. Kochenderfer, “Policy compression for aircraft collision avoidance systems,” in
2016
Earlier work this paper cites.
I. Vasiljevic, A. Chakrabarti, and G. Shakhnarovich, “Examining the impact of blur on recognition by convolutional networks,”
Original
2016
Earlier work this paper cites.
A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,”
Original
2016
Earlier work this paper cites.
A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,”
Original
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in
2016
Earlier work this paper cites.
C. Fefferman, S. Mitter, and H. Narayanan, “Testing the manifold hypothesis,”
2016
Earlier work this paper cites.

Similar

J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in
2016
Cited alongside, same era.
A. Dosovitskiy and T. Brox, “Generating images with perceptual similarity metrics based on deep networks,” in
2016
Cited alongside, same era.
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, and others, “Tensorflow: a system for large-scale machine learning,” in
Original
2016
Cited alongside, same era.
A. Kuehlkamp, B. Becker, and K. Bowyer, “Gender-from-iris or gender-from-mascara?” in
2017
Cited alongside, same era.
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,”
Original
2017
Cited alongside, same era.
N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods,” in
2017
Cited alongside, same era.
——, “Towards evaluating the robustness of neural networks,” in
2017
Cited alongside, same era.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,”
Original
2017
Cited alongside, same era.
L. Engstrom, B. Tran, D. Tsipras, L. Schmidt, and A. Madry, “A rotation and a translation suffice: Fooling cnns with simple transformations,”
Original
2017
Cited alongside, same era.
S. Baluja and I. Fischer, “Adversarial transformation networks: Learning to generate adversarial examples,”
Original
2017
Cited alongside, same era.
A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” in
2017
Cited alongside, same era.
A. Jalal, A. Ilyas, C. Daskalakis, and A. G. Dimakis, “The robust manifold defense: Adversarial training using generative models,”
Original
2017
Cited alongside, same era.

Then

D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry, “Robustness may be at odds with accuracy,”
Original
2018
Later among the works it cites.
S.-M. Moosavi-Dezfooli, A. Fawzi, J. Uesato, and P. Frossard, “Robustness via curvature regularization, and vice versa,”
Original
2018
Later among the works it cites.
C. Kanbak, S.-M. Moosavi-Dezfooli, and P. Frossard, “Geometric robustness of deep networks: analysis and improvement,” in
2018
Later among the works it cites.
Y. Song, R. Shu, N. Kushman, and S. Ermon, “Constructing unrestricted adversarial examples with generative models,” in
2018
Later among the works it cites.
C. Xiao, B. Li, J.-Y. Zhu, W. He, M. Liu, and D. Song, “Generating adversarial examples with adversarial networks,”
Original
2018
Later among the works it cites.
V. Verma, A. Lamb, C. Beckham, A. Najafi, I. Mitliagkas, A. Courville, D. Lopez-Paz, and Y. Bengio, “Manifold mixup: Better representations by interpolating hidden states,”
Original
2018
Later among the works it cites.
I. Higgins, D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner, “Towards a Definition of Disentangled Representations,”
Original
2018
Later among the works it cites.
R. Vershynin,
2018
Later among the works it cites.
B. Recht, R. Roelofs, L. Schmidt, and V. Shankar, “Do imagenet classifiers generalize to imagenet?”
Original
2019
Closest in time.
D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “Natural adversarial examples,”
Original
2019
Closest in time.
A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,”
Original
2019
Closest in time.
T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in
2019
Closest in time.
M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz, “Invariant risk minimization,”
Original
2019
Closest in time.
H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan, “Theoretically principled trade-off between robustness and accuracy,”
Original
2019
Closest in time.
C. Qin, J. Martens, S. Gowal, D. Krishnan, A. Fawzi, S. De, R. Stanforth, P. Kohli
Original
2019
Closest in time.
H. Qiu, C. Xiao, L. Yang, X. Yan, H. Lee, and B. Li, “Semanticadv: Generating adversarial examples via attribute-conditional image editing,”
Original
2019
Closest in time.
S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo, “Cutmix: Regularization strategy to train strong classifiers with localizable features,”
Original
2019
Closest in time.
E. Nalisnick, A. Matsukawa, Y. W. Teh, and B. Lakshminarayanan, “Detecting out-of-distribution inputs to deep generative models using a test for typicality,”
Original
2019
Closest in time.
R. Abdal, Y. Qin, and P. Wonka, “Image2stylegan: How to embed images into the stylegan latent space?”
Original
2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…