Fetching the paper…
Reading the bibliography…
Masked Image Modeling (MIM) methods, like Masked Autoencoders (MAE), efficiently learn a rich representation of the input.
Some methods for classification and analysis of multivariate observations
James B. MacQueen · 1965
Earlier work this paper cites.
Least squares quantization in PCM
Stuart P. Lloyd · 1982
Earlier work this paper cites.
Comparing partitions
Lawrence Hubert and Phipps Arabie · 1985
Earlier work this paper cites.
Silhouettes: a graphical aid to the interpretation and validation of cluster analysis
Peter J. Rousseeuw · 1987
Earlier work this paper cites.
Extracting and composing robust features with denoising autoencoders
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance
Xuan Vinh Nguyen, Julien Epps, and James Bailey · 2010
Earlier work this paper cites.
Web-scale k-means clustering
D. Sculley · 2010
Earlier work this paper cites.
Image clustering using local discriminant models and global integration
Yi Yang, Dong Xu, Feiping Nie, Shuicheng Yan, and Yueting Zhuang · 2010
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
Carl Doersch, Abhinav Gupta, and Alexei A. Efros · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton · 2016
Earlier work this paper cites.
On implementing 2d rectangular assignment algorithms
David Frederic Crouse · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Deep networks with stochastic depth
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinberger · 2016
Earlier work this paper cites.
Context encoders: Feature learning by inpainting
Deepak Pathak, Philipp Krähenbühl, Jeff Donahue, Trevor Darrell, and Alexei A. Efros · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna · 2016
Earlier work this paper cites.
Unsupervised deep embedding for clustering analysis
Junyuan Xie, Ross B. Girshick, and Ali Farhadi · 2016
Earlier work this paper cites.
Deep adaptive image clustering
Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan · 2017
Earlier work this paper cites.
Accurate, large minibatch SGD: training imagenet in 1 hour
Priya Goyal, Piotr Dollár, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He · 2017
Earlier work this paper cites.
Temporal ensembling for semi-supervised learning
Samuli Laine and Timo Aila · 2017
Earlier work this paper cites.
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
Antti Tarvainen and Harri Valpola · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Towards k-means-friendly spaces: Simultaneous deep learning and clustering
Bo Yang, Xiao Fu, Nicholas D. Sidiropoulos, and Mingyi Hong · 2017
Earlier work this paper cites.
Deep clustering for unsupervised learning of visual features
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze · 2018
Earlier work this paper cites.
UMAP: uniform manifold approximation and projection
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance discrimination
Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin · 2018
Earlier work this paper cites.
mixup: Beyond empirical risk minimization
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz · 2018
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
Generalization bounds for unsupervised and semi-supervised learning with autoencoders
Baruch Epstein and Ron Meir · 2019
Earlier work this paper cites.
Learning deep representations by mutual information estimation and maximization
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio · 2019
Earlier work this paper cites.
Invariant information clustering for unsupervised image classification and segmentation
Xu Ji, Andrea Vedaldi, and João F. Henriques · 2019
Cited alongside, same era.
A study of BFLOAT16 for deep learning training
Dhiraj D. Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey · 2019
Cited alongside, same era.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter · 2019
Cited alongside, same era.
Cyanure: An open-source toolbox for empirical risk minimization for python, c++, and soon more
Julien Mairal · 2019
Cited alongside, same era.
Do imagenet classifiers generalize to imagenet?
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar · 2019
Understanding the behaviour of contrastive loss
Feng Wang and Huaping Liu · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny · 2021
Later among the works it cites.
Towards efficient and effective self-supervised learning of visual representations
Sravanti Addepalli, Kaushal Bhogale, Priyam Dey, and R. Venkatesh Babu · 2022
Later among the works it cites.
Masked siamese networks for label-efficient learning
Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas · 2022
Later among the works it cites.
data2vec: A general framework for self-supervised learning in speech, vision and language
Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli · 2022
Later among the works it cites.
Beit: BERT pre-training of image transformers
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
A theoretical analysis of contrastive unsupervised representation learning
Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar · 2019
Cited alongside, same era.
Learning robust global representations by penalizing local predictive power
Haohan Wang, Songwei Ge, Zachary C. Lipton, and Eric P. Xing · 2019
Cited alongside, same era.
Cutmix: Regularization strategy to train strong classifiers with localizable features
Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh, Youngjoon Yoo, and Junsuk Choe · 2019
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin · 2020
Cited alongside, same era.
Generative pretraining from pixels
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever · 2020
Cited alongside, same era.
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton · 2020
Cited alongside, same era.
ELECTRA: pre-training text encoders as discriminators rather than generators
Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning · 2020
Cited alongside, same era.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei · 2022
Later among the works it cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2022
Later among the works it cites.
Semi-supervised vision transformers at scale
Zhaowei Cai, Avinash Ravichandran, Paolo Favaro, Manchen Wang, Davide Modolo, Rahul Bhotika, Zhuowen Tu, and Stefano Soatto · 2022
Later among the works it cites.
Perfectly balanced: Improving transfer and robustness of supervised contrastive learning
Mayee F. Chen, Daniel Y. Fu, Avanika Narayan, Michael Zhang, Zhao Song, Kayvon Fatahalian, and Christopher Ré · 2022
Later among the works it cites.
Flashattention: Fast and memory-efficient exact attention with io-awareness
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré · 2022
Later among the works it cites.
On the strong correlation between model invariance and generalization
Weijian Deng, Stephen Gould, and Liang Zheng · 2022
Later among the works it cites.
Bootstrapped masked autoencoders for vision BERT pretraining
Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, and Nenghai Yu · 2022
Later among the works it cites.
Masked autoencoders as spatiotemporal learners
Christoph Feichtenhofer, Haoqi Fan, Yanghao Li, and Kaiming He · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick · 2022
Later among the works it cites.
Exploring long-sequence masked autoencoders
Ronghang Hu, Shoubhik Debnath, Saining Xie, and Xinlei Chen · 2022
Later among the works it cites.
Masked autoencoders that listen
Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, and Christoph Feichtenhofer · 2022
Later among the works it cites.
Contrastive masked autoencoders are stronger vision learners
Zhicheng Huang, Xiaojie Jin, Chengze Lu, Qibin Hou, Ming-Ming Cheng, Dongmei Fu, Xiaohui Shen, and Jiashi Feng · 2022
Later among the works it cites.
Efficient training of audio transformers with patchout
Khaled Koutini, Jan Schlüter, Hamid Eghbal-zadeh, and Gerhard Widmer · 2022
Later among the works it cites.
On the pros and cons of momentum encoder in self-supervised visual representation learning
Trung Pham, Chaoning Zhang, Axi Niu, Kang Zhang, and Chang D. Yoo · 2022
Later among the works it cites.
Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang · 2022
Later among the works it cites.
Simmim: a simple framework for masked image modeling
Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu · 2022
Later among the works it cites.
Image BERT pre-training with online tokenizer
Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan L. Yuille, and Tao Kong · 2022
Later among the works it cites.
A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions
Sheng Zhou, Hongjia Xu, Zhuonan Zheng, Jiawei Chen, Zhao Li, Jiajun Bu, Jia Wu, Xin Wang, Wenwu Zhu, and Martin Ester · 2022
Later among the works it cites.
The hidden uniform cluster prior in self-supervised learning
Mido Assran, Randall Balestriero, Quentin Duval, Florian Bordes, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael G. Rabbat, and Nicolas Ballas · 2023
Closest in time.
Self-supervised learning from images with a joint-embedding predictive architecture
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael G. Rabbat, Yann LeCun, and Nicolas Ballas · 2023
Closest in time.
A theoretical study of inductive biases in contrastive learning
Jeff Z. HaoChen and Tengyu Ma · 2023
Closest in time.
Layer grafted pre-training: Bridging contrastive learning and masked image modeling for label-efficient representations
Ziyu Jiang, Yinpeng Chen, Mengchen Liu, Dongdong Chen, Xiyang Dai, Lu Yuan, Zicheng Liu, and Zhangyang Wang · 2023
Closest in time.
Infonce loss provably learns cluster-preserving representations
Advait Parulekar, Liam Collins, Karthikeyan Shanmugam, Aryan Mokhtari, and Sanjay Shakkottai · 2023
Closest in time.
The effectiveness of MAE pre-pretraining for billion-scale pretraining
Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan, Vaibhav Aggarwal, Aaron Adcock, Armand Joulin, Piotr Dollár, Christoph Feichtenhofer, Ross B. Girshick, Rohit Girdhar, and Ishan Misra · 2023
Closest in time.
Hubs and hyperspheres: Reducing hubness and improving transductive few-shot learning with hyperspherical embeddings
Daniel J Trosten, Rwiddhi Chakraborty, Sigurd Løkse, Kristoffer Knutsen Wickstrøm, Robert Jenssen, and Michael C Kampffmeyer · 2023
Closest in time.
Teaching matters: Investigating the role of supervision in vision transformers
Matthew Walmer, Saksham Suri, Kamal Gupta, and Abhinav Shrivastava · 2023
Closest in time.
On data scaling in masked image modeling
Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Yixuan Wei, Qi Dai, and Han Hu · 2023
Closest in time.