Fetching the paper…
Reading the bibliography…
Vision transformers have demonstrated remarkable success in a wide range of computer vision tasks over the last years.
A new measure of rank correlation
Maurice G Kendall · 1938
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Estimating or propagating gradients through stochastic neurons for conditional computation
Yoshua Bengio, Nicholas Léonard, and Aaron Courville · 2013
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Reformer: The efficient transformer
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya · 2019
Earlier work this paper cites.
Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov · 2019
Earlier work this paper cites.
Compressing bert: Studying the effects of weight pruning on transfer learning
Mitchell Gordon, Kevin Duh, and Nicholas Andrews · 2020
Earlier work this paper cites.
Power-bert: Accelerating bert inference via progressive word-vector elimination
Saurabh Goyal, Anamitra Roy Choudhury, Saurabh Raje, Venkatesan Chakaravarthy, Yogish Sabharwal, and Ashish Verma · 2020
Earlier work this paper cites.
Softsort: A continuous relaxation for the argsort operator
Sebastian Prillo and Julian Eisenschlos · 2020
Earlier work this paper cites.
Crossvit: Cross-attention multi-scale vision transformer for image classification
Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda · 2021
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2021
Cited alongside, same era.
Localvit: Bringing locality to vision transformers
Yawei Li, Kai Zhang, Jiezhang Cao, Radu Timofte, and Luc Van Gool · 2021
Cited alongside, same era.
Dynamicvit: Efficient vision transformers with dynamic token sparsification
Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh · 2021
Cited alongside, same era.
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou · 2021
Adaptive sparse vit: Towards learnable adaptive token pruning by fully exploiting self-attention
Xiangcheng Liu, Tianyi Wu, and Guodong Guo · 2022
Later among the works it cites.
How to train your vit? data, augmentation, and regularization in vision transformers
Andreas Peter Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer · 2022
Later among the works it cites.
Towards efficient vision transformer inference: a first study of transformers on mobile devices
Xudong Wang, Li Lyna Zhang, Yang Wang, and Mao Yang · 2022
Later among the works it cites.
timm: Pytorch image models, scripts, pretrained weights, and more
Ross Wightman · 2022
Later among the works it cites.
Tinyvit: Fast pretraining distillation for small vision transformers
Kan Wu, Jinnian Zhang, Houwen Peng, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Hardware-aware mobile building block evaluation for computer vision
Maxim Bonnaerens, Matthias Freiberger, Marian Verhelst, and Joni Dambre · 2022
Cited alongside, same era.
Learned token pruning for transformers
Sehoon Kim, Sheng Shen, David Thorsley, Amir Gholami, Woosuk Kwon, Joseph Hassoun, and Kurt Keutzer · 2022
Cited alongside, same era.
Spvit: Enabling faster vision transformers via soft token pruning
Zhenglun Kong, Peiyan Dong, Xiaolong Ma, Xin Meng, Wei Niu, Mengshu Sun, Bin Ren, Minghai Qin, Hao Tang, and Yanzhi Wang · 2022
Cited alongside, same era.
EVit: Expediting vision transformers via token reorganizations
Youwei Liang, Chongjian GE, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie · 2022
Cited alongside, same era.
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo
Cited in the paper.
Post-training quantization for vision transformer
Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao
Cited in the paper.
Evo-vit: Slow-fast token evolution for dynamic vision transformer
Yifan Xu, Zhijie Zhang, Mengdan Zhang, Kekai Sheng, Ke Li, Weiming Dong, Liqing Zhang, Changsheng Xu, and Xing Sun · 2022
Later among the works it cites.
Adavit: Adaptive tokens for efficient vision transformer
Hongxu Yin, Arash Vahdat, Jose Alvarez, Arun Mallya, Jan Kautz, and Pavlo Molchanov · 2022
Later among the works it cites.
Token merging: Your vit but faster
Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman · 2023
Closest in time.