Fetching the paper…
Reading the bibliography…
Contrastive learning typically matches pairs of related views among a number of unrelated negative views.
Augment your batch: better training with larger batches
Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry · 1901
Earlier work this paper cites.
Data-efficient image recognition with contrastive predictive coding
Olivier J. Hénaff, Aravind Srinivas, Jeffrey De Fauw, Ali Razavi, Carl Doersch, S. M. Ali Eslami, and Aäron van den Oord · 1905
Earlier work this paper cites.
Momentum contrast for unsupervised visual representation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick · 1911
Earlier work this paper cites.
Application of the radon-nikodym theorem to the theory of sufficient statistics
Paul R. Halmos and Leonard J. Savage · 1949
Earlier work this paper cites.
The fisher-darmois-koopman-pitman theorem for random processes
Frederick E. Daum · 1986
Earlier work this paper cites.
An application of the principle of maximum information preservation to linear systems
Ralph Linsker · 1988
Earlier work this paper cites.
Improved baselines with momentum contrastive learning
Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He · 2003
Earlier work this paper cites.
Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories
Li Fei-Fei, Robert Fergus, and Pietro Perona · 2005
Earlier work this paper cites.
Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing)
Thomas M. Cover and Joy A. Thomas · 2006
Earlier work this paper cites.
Estimating divergence functionals and the likelihood ratio by convex risk minimization
XuanLong Nguyen, Martin J. Wainwright, and Michael I. Jordan · 2008
Earlier work this paper cites.
Neural approximate sufficient statistics for implicit models
Yanzhi Chen, Dinghuai Zhang, Michael Gutmann, Aaron Courville, and Zhanxing Zhu · 2010
Earlier work this paper cites.
SUN database: Large-scale scene recognition from abbey to zoo
Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba · 2010
Earlier work this paper cites.
Cats and dogs
Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar · 2012
Earlier work this paper cites.
Representation learning: A review and new perspectives
Yoshua Bengio, Aaron C. Courville, and Pascal Vincent · 2013
Earlier work this paper cites.
Collecting a large-scale dataset of fine-grained cars
Jonathan Krause, Jia Deng, Michael Stark, and Li Fei-Fei · 2013
Earlier work this paper cites.
Fine-grained visual classification of aircraft
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew B. Blaschko, and Andrea Vedaldi · 2013
Earlier work this paper cites.
Food-101 - mining discriminative components with random forests
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool · 2014
Earlier work this paper cites.
Describing textures in the wild
Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi · 2014
Earlier work this paper cites.
Cifar-10 (canadian institute for advanced research)
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton · 2014
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, and Li Fei-Fei · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Improved deep metric learning with multi-class n-pair loss objective
Kihyuk Sohn · 2016
Cited alongside, same era.
Deep metric learning via lifted structured feature embedding
Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese · 2016
Cited alongside, same era.
Contrastive-center loss for deep neural networks
Ce Qi and Fei Su · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Scaling SGD batch size to 32k for imagenet training
Yang You, Igor Gitman, and Boris Ginsburg · 2017
Cited alongside, same era.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Tongzhou Wang and Phillip Isola · 2020
Later among the works it cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2021
Later among the works it cites.
Emerging properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin · 2021
Later among the works it cites.
Exploring simple siamese representation learning
Xinlei Chen and Kaiming He · 2021
Later among the works it cites.
An empirical study of training self-supervised vision transformers
Xinlei Chen, Saining Xie, and Kaiming He · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J. Smola · 2017
Cited alongside, same era.
Deep clustering for unsupervised learning of visual features
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze · 2018
Cited alongside, same era.
An efficient framework for learning sentence representations
Lajanugen Logeswaran and Honglak Lee · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Aäron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Cited alongside, same era.
Learning representations by maximizing mutual information across views
Philip Bachman, R. Devon Hjelm, and William Buchwalter · 2019
Cited alongside, same era.
Learning deep representations by mutual information estimation and maximization
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio · 2019
Cited alongside, same era.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter · 2019
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2021
Later among the works it cites.
Drawing multiple augmentation samples per image during training efficiently decreases test error
Stanislav Fort, Andrew Brock, Razvan Pascanu, Soham De, and Samuel L. Smith · 2021
Later among the works it cites.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever · 2021
Later among the works it cites.
Contrastive learning with hard negative samples
Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka · 2021
Later among the works it cites.
Self-supervised learning with data augmentations provably isolates content from style
Julius von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello · 2021
Later among the works it cites.
Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods
Randall Balestriero and Yann LeCun · 2022
Later among the works it cites.
The effects of regularization and data augmentation are class dependent
Randall Balestriero, Léon Bottou, and Yann LeCun · 2022
Later among the works it cites.
Big vision
Lucas Beyer, Xiaohua Zhai, and Alexander Kolesnikov · 2022
Later among the works it cites.
Einops: Clear and reliable tensor manipulations with einstein-like notation
Alex Rogozhnikov · 2022
Later among the works it cites.
Rethinking minimal sufficient representation in contrastive learning
Haoqing Wang, Xun Guo, Zhi-Hong Deng, and Yan Lu · 2022
Later among the works it cites.
Contrastive learning with stronger augmentations
Xiao Wang and Guo-Jun Qi · 2022
Later among the works it cites.
A cookbook of self-supervised learning
Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Grégoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, and Micah Goldblum · 2023
Later among the works it cites.
The role of entropy and reconstruction in multi-view self-supervised learning
Borja Rodríguez Gálvez, Arno Blaas, Pau Rodríguez, Adam Golinski, Xavier Suau, Jason Ramapuram, Dan Busbridge, and Luca Zappella · 2023
Later among the works it cites.
Scorecl: Augmentation-adaptive contrastive learning via score-matching function
Jin Young Kim, Soonwoo Kwon, Hyojun Go, Yunsung Lee, and Seungtaek Choi · 2023
Later among the works it cites.
Towards a rigorous analysis of mutual information in contrastive learning
Kyungeun Lee, Jaeill Kim, Suhyun Kang, and Wonjong Rhee · 2023
Later among the works it cites.
An information-theoretic perspective on variance-invariance-covariance regularization
Ravid Shwartz-Ziv, Randall Balestriero, Kenji Kawaguchi, Tim G. J. Rudner, and Yann LeCun · 2023
Later among the works it cites.
Stabilizing transformer training by preventing attention entropy collapse
Shuangfei Zhai, Tatiana Likhomanenko, Etai Littwin, Dan Busbridge, Jason Ramapuram, Yizhe Zhang, Jiatao Gu, and Joshua M. Susskind · 2023
Later among the works it cites.