2018

Learning to Defend by Learning to Attack

Jiang, Haoming, Chen, Zhehui, Shi, Yuyang et al.

Understand

Adversarial training provides a principled approach for training robust neural networks.

From an optimization perspective, adversarial training is essentially solving a bilevel optimization problem.
The leader problem is trying to learn a robust classifier, while the follower problem is trying to generate adversarial samples.
Unfortunately, such a bilevel problem is difficult to solve due to its highly complicated structure.

Built on

Theoretically principled trade-off between robustness and accuracy
Original
Zhang, H · 1901
Earlier work this paper cites.
On evaluating adversarial robustness
Original
Carlini, N · 1902
Earlier work this paper cites.
Nattack: Learning the distributions of adversarial examples for an improved black-box attack on deep neural networks
Original
Li, Y · 1905
Earlier work this paper cites.
Distributionally robust optimization: A review
Original
Rahimian, H · 1908
Earlier work this paper cites.
Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-… hook
Schmidhuber, J · 1987
Earlier work this paper cites.
A simple weight decay can improve generalization
Krogh, A · 1992
Earlier work this paper cites.
Learning to control fast-weight memories: An alternative to dynamic recurrent networks
Schmidhuber, J · 1992
Earlier work this paper cites.
A neural network that embeds its own meta-levels
Schmidhuber, J · 1993
Earlier work this paper cites.
Learning to learn using gradient descent
Hochreiter, S · 2001
Earlier work this paper cites.
Meta-learning with backpropagation
Younger, A. S · 2001
Earlier work this paper cites.
An overview of bilevel optimization
Colson, B · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A · 2009
Earlier work this paper cites.
Intriguing properties of neural networks
Original
Szegedy, C · 2013
Earlier work this paper cites.
Rich feature hierarchies for accurate object detection and semantic segmentation
Girshick, R · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Kingma, D · 2014
Earlier work this paper cites.

Similar

Deepface: Closing the gap to human-level performance in face verification
Taigman, Y · 2014
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K · 2015
Cited alongside, same era.
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A · 2015
Cited alongside, same era.
Trust region policy optimization
Schulman, J · 2015
Cited alongside, same era.
Learning to learn by gradient descent by gradient descent
Andrychowicz, M · 2016
Cited alongside, same era.
Openai gym
Original
Brockman, G · 2016
Cited alongside, same era.
Learning from conditional distributions via dual embeddings
Original
Dai, B · 2016
Cited alongside, same era.
Distributionally robust stochastic optimization with wasserstein distance
Original
Gao, R · 2016
Cited alongside, same era.
Deep residual learning for image recognition
He, K · 2016
Cited alongside, same era.
Generative adversarial imitation learning
Original
Ho, J · 2016
Cited alongside, same era.
Adversarial machine learning at scale
Original
Kurakin, A · 2016
Cited alongside, same era.
Delving into transferable adversarial examples and black-box attacks
Original
Liu, Y · 2016
Cited alongside, same era.

Then

Adversarial transformation networks: Learning to generate adversarial examples
Original
Baluja, S · 2017
Later among the works it cites.
Towards evaluating the robustness of neural networks
Carlini, N · 2017
Later among the works it cites.
Model-agnostic meta-learning for fast adaptation of deep networks
Original
Finn, C · 2017
Later among the works it cites.
GANs trained by a two time-scale update ReLU converge to a local nash equilibrium
Heusel, M · 2017
Later among the works it cites.
Deep hyperspherical learning
Liu, W · 2017
Later among the works it cites.
Towards deep learning models resistant to adversarial attacks
Original
Madry, A · 2017
Later among the works it cites.
Automatic differentiation in pytorch
Paszke, A · 2017
Later among the works it cites.
mixup: Beyond empirical risk minimization
Original
Zhang, H · 2017
Later among the works it cites.
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Original
Athalye, A · 2018
Closest in time.
Boosting adversarial attacks with momentum
Dong, Y · 2018
Closest in time.
Toward deeper understanding of nonconvex stochastic optimization with momentum using diffusion approximations
Original
Liu, T · 2018
Closest in time.
Spectral normalization for generative adversarial networks
Miyato, T · 2018
Closest in time.
Defense-gan: Protecting classifiers against adversarial attacks using generative models
Original
Samangouei, P · 2018
Closest in time.
Generating adversarial examples with adversarial networks
Original
Xiao, C · 2018
Closest in time.
Adversarial interpolation training: A simple approach for improving model robustness. URL https://openreview.net/pdf?id=Syejj0NYvr
Haichao Zhang, W. X · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…