Fetching the paper…
Reading the bibliography…
Reconstruction and joint embedding have emerged as two leading paradigms in Self Supervised Learning (SSL).
The approximation of one matrix by another of lower rank
Carl Eckart and Gale Young · 1936
Earlier work this paper cites.
Training with noise is equivalent to tikhonov regularization
Chris M Bishop · 1995
Earlier work this paper cites.
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli · 2004
Earlier work this paper cites.
Extracting and composing robust features with denoising autoencoders
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou · 2010
Earlier work this paper cites.
Auto-encoding variational bayes
Diederik P Kingma and Max Welling · 2013
Earlier work this paper cites.
Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets
Evan Z Macosko, Anindita Basu, Rahul Satija, James Nemesh, Karthik Shekhar, Melissa Goldman, Itay Tirosh, Allison R Bialas, Nolan Kamitaki, Emily M Martersteck, et al · 2015
Earlier work this paper cites.
End-to-end optimization of nonlinear transform codes for perceptual quality
Johannes Ballé, Valero Laparra, and Eero P Simoncelli · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Two convolutional neural networks for bird detection in audio signals
Thomas Grill and Jan Schlüter · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
Large batch training of convolutional networks
Yang You, Igor Gitman, and Boris Ginsburg · 2017
Earlier work this paper cites.
Deep learning for classical japanese literature
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
Benchmarking neural network robustness to common corruptions and perturbations
Dan Hendrycks and Thomas Dietterich · 2019
Earlier work this paper cites.
Wide neural networks of any depth evolve as linear models under gradient descent
Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington · 2019
Earlier work this paper cites.
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton · 2020
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Earlier work this paper cites.
Bootstrap your own latent-a new approach to self-supervised learning
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al · 2020
Earlier work this paper cites.
Momentum contrast for unsupervised visual representation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick · 2020
Earlier work this paper cites.
Self-supervised learning of pretext-invariant representations
Ishan Misra and Laurens van der Maaten · 2020
Earlier work this paper cites.
What makes for good views for contrastive learning?
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola · 2020
Earlier work this paper cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Tongzhou Wang and Phillip Isola · 2020
Cited alongside, same era.
Big self-supervised models advance medical image classification
Shekoofeh Azizi, Basil Mustafa, Fiona Ryan, Zachary Beaver, Jan Freyberg, Jonathan Deaton, Aaron Loh, Alan Karthikesalingam, Simon Kornblith, Ting Chen, et al · 2021
Cited alongside, same era.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2021
Cited alongside, same era.
Emerging properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin · 2021
Cited alongside, same era.
An empirical study of training self-supervised vision transformers
Xinlei Chen, Saining Xie, and Kaiming He · 2021
Cited alongside, same era.
Self-supervised learning from images with a joint-embedding predictive architecture
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas · 2023
Later among the works it cites.
A cookbook of self-supervised learning
Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al · 2023
Later among the works it cites.
V-jepa: Latent video prediction for visual representation learning
Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mido Assran, and Nicolas Ballas · 2023
Later among the works it cites.
The ssl interplay: Augmentations, inductive bias, and generalization
Vivien Cabannes, Bobak Kiani, Randall Balestriero, Yann LeCun, and Alberto Bietti · 2023
Later among the works it cites.
Unbiased single-cell morphology with self-supervised vision transformers
Michael Doron, Théo Moutakanni, Zitong S Chen, Nikita Moshkov, Mathilde Caron, Hugo Touvron, Piotr Bojanowski, Wolfgang M Pernice, and Juan C Caicedo · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Provable guarantees for self-supervised deep learning with spectral contrastive loss
Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma · 2021
Cited alongside, same era.
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick · 2021
Cited alongside, same era.
Towards the generalization of contrastive self-supervised learning
Weiran Huang, Mingyang Yi, and Xuyang Zhao · 2021
Cited alongside, same era.
A broad study on the transferability of visual representations with contrastive learning
Ashraful Islam, Chun-Fu Chen, Rameswar Panda, Leonid Karlinsky, Richard Radke, and Rogerio Feris · 2021
Cited alongside, same era.
Understanding dimensional collapse in contrastive self-supervised learning
Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian · 2021
Cited alongside, same era.
Understanding self-supervised learning dynamics without contrastive pairs
Yuandong Tian, Xinlei Chen, and Surya Ganguli · 2021
Cited alongside, same era.
Understanding the behaviour of contrastive loss
Feng Wang and Huaping Liu · 2021
Cited alongside, same era.
Later among the works it cites.
Deep representation learning: Fundamentals, technologies, applications, and open challenges
Amirreza Payandeh, Kourosh T. Baghaei, Pooya Fayyazsanavi, Somayeh Bakhtiari Ramezani, Zhiqian Chen, and Shahram Rahimi · 2023
Later among the works it cites.
On the stepwise nature of self-supervised learning
James B Simon, Maksis Knutins, Liu Ziyin, Daniel Geisz, Abraham J Fetterman, and Joshua Albrecht · 2023
Later among the works it cites.
Learning theory from first principles
Francis Bach · 2024
Later among the works it cites.
How learning by reconstruction produces uninformative features for perception
Randall Balestriero and Yann Lecun · 2024
Later among the works it cites.
A survey on self-supervised learning: Algorithms, applications, and future trends
Jie Gui, Tuo Chen, Jing Zhang, Qiong Cao, Zhenan Sun, Hao Luo, and Dacheng Tao · 2024
Later among the works it cites.
The common stability mechanism behind most self-supervised learning approaches
Abhishek Jha, Matthew B Blaschko, Yuki M Asano, and Tinne Tuytelaars · 2024
Later among the works it cites.
Bidur Khanal, Binod Bhattarai, Bishesh Khanal, and Cristian Linte · 2024
Later among the works it cites.
Bipasha Kundu, Bidur Khanal, Richard Simon, and Cristian A Linte · 2024
Later among the works it cites.
The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective
Chi-Heng Lin, Chiraag Kaushik, Eva L Dyer, and Vidya Muthukumar · 2024
Later among the works it cites.
How jepa avoids noisy features: The implicit bias of deep linear self distillation networks
Etai Littwin, Omid Saremi, Madhu Advani, Vimal Thilak, Preetum Nakkiran, Chen Huang, and Joshua Susskind · 2024
Later among the works it cites.
Augmentations vs algorithms: What works in self-supervised learning
Warren Morningstar, Alex Bijamov, Chris Duvarney, Luke Friedman, Neha Kalibhat, Luyang Liu, Philip Mansfield, Renan Rojas-Gomez, Karan Singhal, Bradley Green, et al · 2024
Later among the works it cites.
Virchow2: Scaling self-supervised mixed magnification models in pathology
Eric Zimmermann, Eugene Vorontsov, Julian Viret, Adam Casson, Michal Zelechowski, George Shaikovski, Neil Tenenholtz, James Hall, David Klimstra, Razik Yousfi, et al · 2024
Later among the works it cites.
How much semantic information is available in large language model tokens?
David A Haslett and Zhenguang G Cai · 2025
Closest in time.
An empirically grounded identifiability theory will accelerate self-supervised learning research
Patrik Reizinger, Randall Balestriero, David Klindt, and Wieland Brendel · 2025
Closest in time.
Panopticon: Advancing any-sensor foundation models for earth observation
Leonard Waldmann, Ando Shah, Yi Wang, Nils Lehmann, Adam J Stewart, Zhitong Xiong, Xiao Xiang Zhu, Stefan Bauer, and John Chuang · 2025
Closest in time.
Word form matters: Llms’ semantic reconstruction under typoglycemia
Chenxi Wang, Tianle Gu, Zhongyu Wei, Lang Gao, Zirui Song, and Xiuying Chen · 2025
Closest in time.
Ai-powered virtual tissues from spatial proteomics for clinical diagnostics and biomedical discovery
Johann Wenckstern, Eeshaan Jain, Kiril Vasilev, Matteo Pariset, Andreas Wicki, Gabriele Gut, and Charlotte Bunne · 2025
Closest in time.
Learning from models beyond fine-tuning
Hongling Zheng, Li Shen, Anke Tang, Yong Luo, Han Hu, Bo Du, Yonggang Wen, and Dacheng Tao · 2025
Closest in time.