Fetching the paper…
Reading the bibliography…
In self-supervised representation learning, a common idea behind most of the state-of-the-art approaches is to enforce the robustness of the representations to predefined augmentations.
The “independent components” of natural scenes are edge filters
Anthony J. Bell and Terrence J. Sejnowski · 1997
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
An analysis of single-layer networks in unsupervised feature learning
Adam Coates, Andrew Ng, and Honglak Lee · 2011
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al · 2015
Earlier work this paper cites.
Reducing overfitting in deep networks by decorrelating representations
Michael Cogswell, Faruk Ahmed, Ross B. Girshick, Larry Zitnick, and Dhruv Batra · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts
Ilya Loshchilov and Frank Hutter · 2016
Earlier work this paper cites.
Achieving human parity in conversational speech recognition
Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig · 2016
Earlier work this paper cites.
Regularizing deep convolutional neural networks with a structured decorrelation constraint
Wei Xiong, Bo Du, Lefei Zhang, Ruimin Hu, and Dacheng Tao · 2016
Earlier work this paper cites.
Accurate, large minibatch sgd: Training imagenet in 1 hour
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He · 2017
Earlier work this paper cites.
Large batch training of convolutional networks
Yang You, Igor Gitman, and Boris Ginsburg · 2017
Earlier work this paper cites.
Deep clustering for unsupervised learning of visual features
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze · 2018
Earlier work this paper cites.
Decorrelated batch normalization
Lei Huang, Dawei Yang, Bo Lang, and Jia Deng · 2018
Earlier work this paper cites.
Representation learning with contrastive predictive coding
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance-level discrimination
Zhirong Wu, Yuanjun Xiong, Stella Yu, and Dahua Lin · 2018
Earlier work this paper cites.
Unsupervised pre-training of image features on non-curated data
Mathilde Caron, Piotr Bojanowski, Julien Mairal, and Armand Joulin · 2019
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Learning deep representations by mutual information estimation and maximization
R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio · 2019
Cited alongside, same era.
Iterative normalization: Beyond standardization towards efficient whitening
Lei Huang, Yi Zhou, Fan Zhu, Li Liu, and Ling Shao · 2019
Cited alongside, same era.
Unsupervised domain adaptation using feature-whitening and consensus loss
Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, and Elisa Ricci · 2019
Cited alongside, same era.
Whitening and coloring batch transform for gans
Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe · 2019
Cited alongside, same era.
Local aggregation for unsupervised learning of visual embeddings
Data-efficient image recognition with contrastive predictive coding
Olivier Henaff · 2020
Later among the works it cites.
Contrastive learning with adversarial examples
Chih-Hui Ho and Nuno Vasconcelos · 2020
Later among the works it cites.
Qianjiang Hu, Xiao Wang, Wei Hu, and Guo-Jun Qi · 2020
Later among the works it cites.
Boosting contrastive self-supervised learning with false negative cancellation
Tri Huynh, Simon Kornblith, Matthew R. Walter, Michael Maire, and Maryam Khademi · 2020
Later among the works it cites.
Hard negative mixing for contrastive learning
Yannis Kalantidis, Mert Bülent Sariyildiz, Noé Pion, Philippe Weinzaepfel, and Diane Larlus · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Chengxu Zhuang, Alex Lin Zhai, and Daniel Yamins · 2019
Cited alongside, same era.
Self-labelling via simultaneous clustering and representation learning
Yuki Markus Asano, Christian Rupprecht, and Andrea Vedaldi · 2020
Cited alongside, same era.
Parametric instance classification for unsupervised visual feature learning
Yue Cao, Zhenda Xie, Bin Liu, Yutong Lin, Zheng Zhang, and Han Hu · 2020
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin · 2020
Cited alongside, same era.
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton · 2020
Cited alongside, same era.
Improved baselines with momentum contrastive learning
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He · 2020
Cited alongside, same era.
Exploring simple siamese representation learning
Xinlei Chen and Kaiming He · 2020
Cited alongside, same era.
Chunyuan Li, Xiujun Li, Lei Zhang, Baolin Peng, Mingyuan Zhou, and Jianfeng Gao · 2020
Later among the works it cites.
Prototypical contrastive learning of unsupervised representations
Junnan Li, Pan Zhou, Caiming Xiong, Richard Socher, and Steven CH Hoi · 2020
Later among the works it cites.
Selfaugment: Automatic augmentation policies for self-supervised learning, 2020
Colorado Reed, Sean Metzger, Aravind Srinivas, Trevor Darrell, and Kurt Keutzer · 2020
Later among the works it cites.
Byol works even without batch statistics
Pierre H Richemond, Jean-Bastien Grill, Florent Altché, Corentin Tallec, Florian Strub, Andrew Brock, Samuel Smith, Soham De, Razvan Pascanu, Bilal Piot, et al · 2020
Later among the works it cites.
Contrastive learning with hard negative samples
Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka · 2020
Later among the works it cites.
What makes for good views for contrastive learning
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola · 2020
Later among the works it cites.
Unsupervised representation learning by invariance propagation
Feng Wang, Huaping Liu, Di Guo, and Fuchun Sun · 2020
Later among the works it cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Tongzhou Wang and Phillip Isola · 2020
Later among the works it cites.
Mine your own view: Self-supervised learning through across-sample prediction
Mehdi Azabou, Mohammad Gheshlaghi Azar, Ran Liu, Chi-Heng Lin, Erik C. Johnson, Kiran Bhaskaran-Nair, Max Dabagia, Keith B. Hengen, William R. Gray Roncal, Michal Valko, and Eva L. Dyer · 2021
Closest in time.
Understanding self-supervised learning dynamics without contrastive pairs
Yuandong Tian, Xinlei Chen, and Surya Ganguli · 2021
Closest in time.
Seed the views: Hierarchical semantic alignment for contrastive representation learning, 2021
Haohang Xu, Xiaopeng Zhang, Hao Li, Lingxi Xie, Hongkai Xiong, and Qi Tian · 2021
Closest in time.
Barlow twins: Self-supervised learning via redundancy reduction
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny · 2021
Closest in time.