Fetching the paper…
Reading the bibliography…
Masked Autoencoders (MAE) have been popular paradigms for large-scale vision representation pre-training.
Histograms of oriented gradients for human detection , Vol. 1, 886–893 (Ieee, 2005)
Dalal, N. & Triggs, B · 2005
Earlier work this paper cites.
Microsoft coco: Common objects in context , 740–755 (Springer, 2014)
Lin, T.-Y. et al · 2014
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Russakovsky, O. et al · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., Dean, J. et al · 2015
Earlier work this paper cites.
Mask r-cnn , 2961–2969 (2017)
He, K., Gkioxari, G., Dollár, P. & Girshick, R · 2017
Earlier work this paper cites.
Feature pyramid networks for object detection , 2117–2125 (2017)
Lin, T.-Y. et al · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance discrimination , 3733–3742 (2018)
Wu, Z., Xiong, Y., Yu, S. X. & Lin, D · 2018
Earlier work this paper cites.
Fixing weight decay regularization in adam (2018)
Loshchilov, I. & Hutter, F · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A. et al · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T. et al · 2020
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A. et al · 2020
Earlier work this paper cites.
Momentum contrast for unsupervised visual representation learning , 9729–9738 (2020)
He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R · 2020
Earlier work this paper cites.
A simple framework for contrastive learning of visual representations , 1597–1607 (PMLR, 2020)
Chen, T., Kornblith, S., Norouzi, M. & Hinton, G · 2020
Earlier work this paper cites.
Learning transferable visual models from natural language supervision , 8748–8763 (PMLR, 2021)
Radford, A. et al · 2021
Earlier work this paper cites.
Emerging properties in self-supervised vision transformers , 9650–9660 (2021)
Caron, M. et al · 2021
Earlier work this paper cites.
Beit: Bert pre-training of image transformers
Bao, H., Dong, L. & Wei, F · 2021
Earlier work this paper cites.
Zero-shot text-to-image generation , 8821–8831 (PMLR, 2021)
Ramesh, A. et al · 2021
Cited alongside, same era.
Swin transformer: Hierarchical vision transformer using shifted windows , 10012–10022 (2021)
Liu, Z. et al · 2021
Cited alongside, same era.
Container: Context aggregation network
Gao, P., Lu, J., Li, H., Mottaghi, R. & Kembhavi, A · 2021
Cited alongside, same era.
Early convolutions help transformers see better
Xiao, T. et al · 2021
Cited alongside, same era.
Mst: Masked self-supervised transformer for visual representation
Li, Z. et al · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision (2021)
Radford, A. et al · 2021
Masked autoencoders enable efficient knowledge distillers
Bai, Y. et al · 2022
Later among the works it cites.
Convmae: Masked convolution meets masked autoencoders
Gao, P., Ma, T., Li, H., Dai, J. & Qiao, Y · 2022
Later among the works it cites.
Uniform masking: Enabling mae pre-training for pyramid-based vision transformers with locality
Li, X., Wang, W., Yang, L. & Yang, J · 2022
Later among the works it cites.
Mixmim: Mixed and masked image modeling for efficient visual representation learning
Liu, J., Huang, X., Liu, Y. & Li, H · 2022
Later among the works it cites.
Green hierarchical vision transformer for masked image modeling
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Benchmarking detection transfer learning with vision transformers
Li, Y. et al · 2021
Cited alongside, same era.
Masked autoencoders are scalable vision learners , 16000–16009 (2022)
He, K. et al · 2022
Cited alongside, same era.
Masked feature prediction for self-supervised visual pre-training , 14668–14678 (2022)
Wei, C. et al · 2022
Cited alongside, same era.
Data2vec: A general framework for self-supervised learning in speech, vision and language
Baevski, A. et al · 2022
Cited alongside, same era.
Mvp: Multimodality-guided visual pre-training
Wei, L., Xie, L., Zhou, W., Li, H. & Tian, Q · 2022
Cited alongside, same era.
Beit v2: Masked image modeling with vector-quantized visual tokenizers
Peng, Z., Dong, L., Bao, H., Ye, Q. & Wei, F · 2022
Cited alongside, same era.
Huang, L. et al · 2022
Later among the works it cites.
Uniformer: Unifying convolution and self-attention for visual recognition
Li, K. et al · 2022
Later among the works it cites.
Adversarial masking for self-supervised learning , 20026–20040 (PMLR, 2022)
Shi, Y., Siddharth, N., Torr, P. & Kosiorek, A. R · 2022
Later among the works it cites.
What to hide from your students: Attention-guided masked image modeling
Kakogeorgiou, I. et al · 2022
Later among the works it cites.
Context autoencoder for self-supervised representation learning
Chen, X. et al · 2022
Later among the works it cites.
Simmim: A simple framework for masked image modeling , 9653–9663 (2022)
Xie, Z. et al · 2022
Later among the works it cites.
Contrastive masked autoencoders are stronger vision learners
Huang, Z. et al · 2022
Later among the works it cites.
Revisiting over-smoothing in bert from the perspective of graph
Shi, H. et al · 2022
Later among the works it cites.
Repre: Improving self-supervised vision transformer with reconstructive pre-training
Wang, L. et al · 2022
Later among the works it cites.
Cmt: Convolutional neural networks meet vision transformers , 12175–12185 (2022)
Guo, J. et al · 2022
Later among the works it cites.
Exploring plain vision transformer backbones for object detection
Li, Y., Mao, H., Girshick, R. & He, K · 2022
Later among the works it cites.
Dino: Detr with improved denoising anchor boxes for end-to-end object detection
Zhang, H. et al · 2022
Later among the works it cites.