Fetching the paper…
Reading the bibliography…
In this paper, we ask whether Vision Transformers (ViTs) can serve as an underlying architecture for improving the adversarial robustness of machine learning models against evasion attacks.
“Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories”
Li Fei-Fei, Rob Fergus and Pietro Perona · 2004
Earlier work this paper cites.
“Automated Flower Classification over a Large Number of Classes”
M-E. Nilsback and A. Zisserman · 2008
Earlier work this paper cites.
“ImageNet: A Large-Scale Hierarchical Image Database”
J. Deng et al · 2009
Earlier work this paper cites.
“Learning multiple layers of features from tiny images”, 2009
Alex Krizhevsky · 2009
Earlier work this paper cites.
“Evasion attacks against machine learning at test time”
Battista Biggio et al · 2013
Earlier work this paper cites.
“Intriguing properties of neural networks”
Christian Szegedy et al · 2013
Earlier work this paper cites.
“Neural machine translation by jointly learning to align and translate”
Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio · 2014
Earlier work this paper cites.
“Deep residual learning for image recognition. arXiv 2015”
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun · 2015
Earlier work this paper cites.
“Gaussian error linear units (gelus)”
Dan Hendrycks and Kevin Gimpel · 2016
Earlier work this paper cites.
“Towards deep learning models resistant to adversarial attacks”
Aleksander Madry et al · 2017
Earlier work this paper cites.
“Attention is all you need”
Ashish Vaswani et al · 2017
Earlier work this paper cites.
“mixup: Beyond empirical risk minimization”
Hongyi Zhang, Moustapha Cisse, Yann Dauphin and David Lopez-Paz · 2017
Earlier work this paper cites.
“Revisiting unreasonable effectiveness of data in deep learning era”
Chen Sun, Abhinav Shrivastava, Saurabh Singh and Abhinav Gupta · 2017
Earlier work this paper cites.
“Decoupled weight decay regularization”
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
“Robustness may be at odds with accuracy”
Dimitris Tsipras et al · 2018
Earlier work this paper cites.
“Adversarially robust generalization requires more data”
Ludwig Schmidt et al · 2018
Earlier work this paper cites.
“Sigmoid-weighted linear units for neural network function approximation in reinforcement learning”
Stefan Elfwing, Eiji Uchibe and Kenji Doya · 2018
Earlier work this paper cites.
“Theoretically principled trade-off between robustness and accuracy”
Hongyang Zhang et al · 2019
Earlier work this paper cites.
“Adversarial robustness as a prior for learned representations”
Logan Engstrom et al · 2019
Earlier work this paper cites.
“PyTorch: An Imperative Style, High-Performance Deep Learning Library”
Adam Paszke et al · 2019
Earlier work this paper cites.
“Cutmix: Regularization strategy to train strong classifiers with localizable features”
Sangdoo Yun et al · 2019
Earlier work this paper cites.
“Robustness (Python Library)”, 2019
Logan Engstrom et al · 2019
Earlier work this paper cites.
“Using Pre-Training Can Improve Model Robustness and Uncertainty”
Dan Hendrycks, Kimin Lee and Mantas Mazeika · 2019
Earlier work this paper cites.
“PyTorch Image Models”
Ross Wightman · 2019
Earlier work this paper cites.
“Adversarial training for free!”
Ali Shafahi et al · 2019
Earlier work this paper cites.
“Unlabeled data improves adversarial robustness”
Yair Carmon et al · 2019
Earlier work this paper cites.
Cihang Xie et al · 2020
Cited alongside, same era.
“Adversarial vertex mixup: Toward better adversarially robust generalization”
Saehyung Lee, Hyungyu Lee and Sungroh Yoon · 2020
Cited alongside, same era.
“Improving Adversarial Robustness Requires Revisiting Misclassified Examples”
Yisen Wang et al · 2020
Cited alongside, same era.
“Adversarial weight perturbation helps robust generalization”
Dongxian Wu, Shu-Tao Xia and Yisen Wang · 2020
Cited alongside, same era.
“Robustbench: a standardized adversarial robustness benchmark”
Francesco Croce et al · 2020
Cited alongside, same era.
“Overfitting in adversarially robust deep learning”
“Reveal of vision transformers robustness against adversarial attacks”
Ahmed Aldahdooh, Wassim Hamidouche and Olivier Deforges · 2021
Later among the works it cites.
“On the Adversarial Robustness of Vision Transformers”
Rulin Shao et al · 2021
Later among the works it cites.
“Transformers in vision: A survey”
Salman Khan et al · 2021
Later among the works it cites.
“XCiT: Cross-Covariance Image Transformers”
Alaaeldin El-Nouby et al · 2021
Later among the works it cites.
“Training data-efficient image transformers & distillation through attention”
Hugo Touvron et al · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Leslie Rice, Eric Wong and Zico Kolter · 2020
Cited alongside, same era.
“Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks”
Francesco Croce and Matthias Hein · 2020
Cited alongside, same era.
“Randaugment: Practical automated data augmentation with a reduced search space”
Ekin Cubuk, Barret Zoph, Jonathon Shlens and Quoc Le · 2020
Cited alongside, same era.
“Random erasing data augmentation”
Zhun Zhong et al · 2020
Cited alongside, same era.
“Bag of tricks for adversarial training”
Tianyu Pang et al · 2020
Cited alongside, same era.
“Do adversarially robust imagenet models transfer better?”
Hadi Salman et al · 2020
Cited alongside, same era.
“Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack”
Francesco Croce and Matthias Hein · 2020
Cited alongside, same era.
Hugo Touvron et al · 2021
Later among the works it cites.
“MetaFormer is Actually What You Need for Vision”
Weihao Yu et al · 2021
Later among the works it cites.
“LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference”
Benjamin Graham et al · 2021
Later among the works it cites.
“Helper-based Adversarial Training: Reducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off”
Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli · 2021
Later among the works it cites.
“Resnet strikes back: An improved training procedure in timm”
Ross Wightman, Hugo Touvron and Hervé Jégou · 2021
Later among the works it cites.
“RobustBench website” [Accessed Sep 1, 2022], 2021
Francesco Croce et al · 2021
Later among the works it cites.
“Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows”
Ze Liu et al · 2021
Later among the works it cites.
“Adversarial robustness comparison of vision transformer and mlp-mixer to cnns”
Philipp Benz et al · 2021
Later among the works it cites.
“On the robustness of vision transformers to adversarial examples”
Kaleel Mahmood, Rigel Mahmood and Marten Van · 2021
Later among the works it cites.
“Do wider neural networks really help adversarial robustness?”
Boxi Wu et al · 2021
Later among the works it cites.
“Pyramid Adversarial Training Improves ViT Performance”
Charles Herrmann et al · 2021
Later among the works it cites.
“Parameterizing activation functions for adversarial robustness”
Sihui Dai, Saeed Mahloujifar and Prateek Mittal · 2022
Closest in time.
“Towards Efficient Adversarial Training on Vision Transformers”
Boxi Wu et al · 2022
Closest in time.
URL: https://paperswithcode.com/sota/image-classification-on-imagenet
“Image Classification on ImageNet (Papers with Code)” [Accessed Sep 1, 2022], 2022 · 2022
Closest in time.
“How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers”
Andreas Steiner et al · 2022
Closest in time.
“A ConvNet for the 2020s”
Zhuang Liu et al · 2022
Closest in time.
“EasyRobust: A Large-scale Robust Training Toolkit” [Online; accessed 24-August-2022], https://github.com/alibaba/easyrobust/tree/4c0ec0b0c908004b5c65f718de43a530a0856366 , 2022
Xiaofeng Mao et al · 2022
Closest in time.
“BEiT: BERT Pre-Training of Image Transformers”
Hangbo Bao, Li Dong, Songhao Piao and Furu Wei · 2022
Closest in time.
“An Impartial Take to the CNN vs Transformer Robustness Contest”
Francesco Pinto, Philip Torr and Puneet Dokania · 2022
Closest in time.
“Towards robust vision transformer”
Xiaofeng Mao et al · 2022
Closest in time.