Fetching the paper…
Reading the bibliography…
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning.
Model-based reinforcement learning for atari
L. Kaiser, M. Babaeizadeh, P. Milos, B. Osinski, R. H. Campbell, K. Czechowski, D. Erhan, C. Finn, P. Kozakowski, S. Levine, A. Mohiuddin, R. Sepassi, G. Tucker, and H. Michalewski · 1903
Earlier work this paper cites.
Mastering atari, go, chess and shogi by planning with a learned model
J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, and D. Silver · 1911
Earlier work this paper cites.
Estimating optimal transformations for multiple regression and correlation
L. Breiman and J. H. Friedman · 1985
Earlier work this paper cites.
Relations between two sets of variates
H. Hotelling · 1992
Earlier work this paper cites.
Signature verification using a" siamese" time delay neural network
J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah · 1993
Earlier work this paper cites.
Nonlinear canonical correlation analysis by neural networks
W. W. Hsieh · 2000
Earlier work this paper cites.
Quick training of probabilistic neural nets by importance sampling
Y. Bengio and J.-S. Senécal · 2003
Earlier work this paper cites.
Improved baselines with momentum contrastive learning
X. Chen, H. Fan, R. Girshick, and K. He · 2003
Earlier work this paper cites.
Neighbourhood components analysis
J. Goldberger, G. E. Hinton, S. Roweis, and R. R. Salakhutdinov · 2004
Earlier work this paper cites.
Curl: Contrastive unsupervised representations for reinforcement learning
A. Srinivas, M. Laskin, and P. Abbeel · 2004
Earlier work this paper cites.
Maximum margin clustering
L. Xu, J. Neufeld, B. Larson, and D. Schuurmans · 2004
Earlier work this paper cites.
Language models are few-shot learners, 2020
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei · 2005
Earlier work this paper cites.
Learning a similarity metric discriminatively, with application to face verification
S. Chopra, R. Hadsell, and Y. LeCun · 2005
Earlier work this paper cites.
Greedy layer-wise training of deep networks
Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle · 2006
Earlier work this paper cites.
Big self-supervised models are strong semi-supervised learners
T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. Hinton · 2006
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
R. Hadsell, S. Chopra, and Y. LeCun · 2006
Earlier work this paper cites.
A fast learning algorithm for deep belief nets
G. E. Hinton, S. Osindero, and Y.-W. Teh · 2006
Earlier work this paper cites.
Linformer: Self-attention with linear complexity
S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma · 2006
Earlier work this paper cites.
Learning invariant representations for reinforcement learning without reconstruction
A. Zhang, R. McAllister, R. Calandra, Y. Gal, and S. Levine · 2006
Earlier work this paper cites.
Data-efficient reinforcement learning with self-predictive representations
M. Schwarzer, A. Anand, R. Goel, R. D. Hjelm, A. Courville, and P. Bachman · 2007
Earlier work this paper cites.
Adaptive importance sampling to accelerate training of a neural probabilistic language model
Y. Bengio and J.-S. Senécal · 2008
Earlier work this paper cites.
Extracting and composing robust features with denoising autoencoders
P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Overview of supervised learning
T. Hastie, R. Tibshirani, and J. Friedman · 2009
Earlier work this paper cites.
Distance metric learning for large margin nearest neighbor classification
K. Q. Weinberger and L. K. Saul · 2009
Earlier work this paper cites.
Large scale online learning of image similarity through ranking
G. Chechik, V. Sharma, U. Shalit, and S. Bengio · 2010
Earlier work this paper cites.
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
M. Gutmann and A. Hyvärinen · 2010
Earlier work this paper cites.
Discriminative clustering for image co-segmentation
A. Joulin, F. Bach, and J. Ponce · 2010
Earlier work this paper cites.
Understanding self-supervised learning with dual deep networks
Y. Tian, L. Yu, X. Chen, and S. Ganguli · 2010
Earlier work this paper cites.
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou · 2010
Earlier work this paper cites.
Exploring simple siamese representation learning
X. Chen and K. He · 2011
Earlier work this paper cites.
A fast and simple algorithm for training neural probabilistic language models
A. Mnih and Y. W. Teh · 2012
Earlier work this paper cites.
Deep canonical correlation analysis
G. Andrew, R. Arora, J. Bilmes, and K. Livescu · 2013
Earlier work this paper cites.
Auto-encoding variational bayes
D. P. Kingma and M. Welling · 2013
Earlier work this paper cites.
Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks
D.-H. Lee et al · 2013
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean · 2013
Earlier work this paper cites.
Learning word embeddings efficiently with noise-contrastive estimation
A. Mnih and K. Kavukcuoglu · 2013
Earlier work this paper cites.
Playing atari with deep reinforcement learning
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller · 2013
Earlier work this paper cites.
Discriminative unsupervised feature learning with convolutional neural networks
A. Dosovitskiy, J. T. Springenberg, M. Riedmiller, and T. Brox · 2014
Earlier work this paper cites.
Notes on noise contrastive estimation and negative sampling
C. Dyer · 2014
Earlier work this paper cites.
Depth map prediction from a single image using a multi-scale deep network
D. Eigen, C. Puhrsch, and R. Fergus · 2014
Earlier work this paper cites.
Generative adversarial networks
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Earlier work this paper cites.
On using very large target vocabulary for neural machine translation
S. Jean, K. Cho, R. Memisevic, and Y. Bengio · 2014
Earlier work this paper cites.
Proximal algorithms
N. Parikh, S. Boyd, et al · 2014
Earlier work this paper cites.
How transferable are features in deep neural networks?
J. Yosinski, J. Clune, Y. Bengio, and H. Lipson · 2014
Earlier work this paper cites.
Learning deep features for scene recognition using places database
B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva · 2014
Earlier work this paper cites.
Learning to see by moving
P. Agrawal, J. Carreira, and J. Malik · 2015
Earlier work this paper cites.
Discriminative unsupervised feature learning with exemplar convolutional neural networks
D. Alexey, P. Fischer, J. Tobias, M. R. Springenberg, and T. Brox · 2015
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
C. Doersch, A. Gupta, and A. A. Efros · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
G. Hinton, O. Vinyals, J. Dean, et al · 2015
Earlier work this paper cites.
An efficient algorithm for information decomposition and extraction
A. Makur, F. Kozynski, S.-L. Huang, and L. Zheng · 2015
Earlier work this paper cites.
Facenet: A unified embedding for face recognition and clustering
F. Schroff, D. Kalenichenko, and J. Philbin · 2015
Earlier work this paper cites.
Unsupervised and semi-supervised learning with categorical generative adversarial networks
J. T. Springenberg · 2015
Earlier work this paper cites.
On deep multi-view representation learning
W. Wang, R. Arora, K. Livescu, and J. Bilmes · 2015
Earlier work this paper cites.
Unsupervised learning of visual representations using videos
X. Wang and A. Gupta · 2015
Earlier work this paper cites.
Deep learning , volume 1
I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio · 2016
Earlier work this paper cites.
Deep networks with stochastic depth
G. Huang, Y. Sun, Z. Liu, D. Sedra, and K. Q. Weinberger · 2016
Earlier work this paper cites.
Unsupervised feature extraction by time-contrastive learning and nonlinear ica
A. Hyvarinen and H. Morioka · 2016
Earlier work this paper cites.
Learning representations for automatic colorization
G. Larsson, M. Maire, and G. Shakhnarovich · 2016
Earlier work this paper cites.
Unsupervised learning of visual representations by solving jigsaw puzzles
M. Noroozi and P. Favaro · 2016
Earlier work this paper cites.
Deep metric learning via lifted structured feature embedding
H. Oh Song, Y. Xiang, S. Jegelka, and S. Savarese · 2016
Earlier work this paper cites.
Wavenet: A generative model for raw audio
A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu · 2016
Earlier work this paper cites.
Ambient sound provides supervision for visual learning
A. Owens, J. Wu, J. H. McDermott, W. T. Freeman, and A. Torralba · 2016
Earlier work this paper cites.
Context encoders: Feature learning by inpainting
D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros · 2016
Earlier work this paper cites.
Improved techniques for training gans
T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen · 2016
Earlier work this paper cites.
Improved deep metric learning with multi-class n-pair loss objective
K. Sohn · 2016
Earlier work this paper cites.
Colorful image colorization
R. Zhang, P. Isola, and A. A. Efros · 2016
Earlier work this paper cites.
Unsupervised learning by predicting noise
P. Bojanowski and A. Joulin · 2017
Earlier work this paper cites.
Good semi-supervised learning that requires a bad gan
Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. R. Salakhutdinov · 2017
Earlier work this paper cites.
Adversarial feature learning
J. Donahue, P. Krähenbühl, and T. Darrell · 2017
Earlier work this paper cites.
Accurate, large minibatch sgd: Training imagenet in 1 hour
P. Goyal, P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He · 2017
Earlier work this paper cites.
Mask r-cnn
K. He, G. Gkioxari, P. Dollár, and R. Girshick · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
I. Loshchilov and F. Hutter · 2017
Earlier work this paper cites.
A laplacian framework for option discovery in reinforcement learning
M. C. Machado, M. G. Bellemare, and M. Bowling · 2017
Earlier work this paper cites.
Representation learning by learning to count
M. Noroozi, H. Pirsiavash, and P. Favaro · 2017
Earlier work this paper cites.
Learning features by watching objects move
D. Pathak, R. Girshick, P. Dollár, T. Darrell, and B. Hariharan · 2017
Earlier work this paper cites.
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
A. Tarvainen and H. Valpola · 2017
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
Large batch training of convolutional networks
Y. You, I. Gitman, and B. Ginsburg · 2017
Earlier work this paper cites.
Split-brain autoencoders: Unsupervised learning by cross-channel prediction
R. Zhang, P. Isola, and A. A. Efros · 2017
Earlier work this paper cites.
Unsupervised learning of depth and ego-motion from video
T. Zhou, M. Brown, N. Snavely, and D. G. Lowe · 2017
Earlier work this paper cites.
Mutual information neural estimation
M. I. Belghazi, A. Baratin, S. Rajeshwar, S. Ozair, Y. Bengio, A. Courville, and D. Hjelm · 2018
Earlier work this paper cites.
Deep clustering for unsupervised learning of visual features
M. Caron, P. Bojanowski, A. Joulin, and M. Douze · 2018
Earlier work this paper cites.
Conditional noise-contrastive estimation of unnormalised models
C. Ceylan and M. U. Gutmann · 2018
Earlier work this paper cites.
Unsupervised cross-modal alignment of speech and text embedding spaces
Y.-A. Chung, W.-H. Weng, S. Tong, and J. Glass · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova · 2018
Earlier work this paper cites.
Born again neural networks
T. Furlanello, Z. Lipton, M. Tschannen, L. Itti, and A. Anandkumar · 2018
Earlier work this paper cites.
Unsupervised representation learning by predicting image rotations
S. Gidaris, P. Singh, and N. Komodakis · 2018
Earlier work this paper cites.
The inaturalist species classification and detection dataset
G. V. Horn, O. M. Aodha, Y. Song, Y. Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie · 2018
Earlier work this paper cites.
Z. Ma and M. Collins · 2018
Earlier work this paper cites.
Virtual adversarial training: a regularization method for supervised and semi-supervised learning
T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii · 2018
Earlier work this paper cites.
Boosting self-supervised learning via knowledge transfer
M. Noroozi, A. Vinjimoor, P. Favaro, and H. Pirsiavash · 2018
Earlier work this paper cites.
Representation learning with contrastive predictive coding
A. v. d. Oord, Y. Li, and O. Vinyals · 2018
Cited alongside, same era.
Improving language understanding by generative pre-training
A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, et al · 2018
Cited alongside, same era.
Spreading vectors for similarity search
A. Sablayrolles, M. Douze, C. Schmid, and H. Jégou · 2018
Cited alongside, same era.
Time-contrastive networks: Self-supervised learning from video
P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, S. Levine, and G. Brain · 2018
Cited alongside, same era.
Deep graph infomax, 2018
P. Veličković, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm · 2018
Cited alongside, same era.
Understanding self-supervised learning dynamics without contrastive pairs
Y. Tian, X. Chen, and S. Ganguli · 2021
Later among the works it cites.
Subtab: Subsetting features of tabular data for self-supervised representation learning
T. Ucar, E. Hajiramezanali, and L. Edwards · 2021
Later among the works it cites.
Instance localization for self-supervised detection pretraining
C. Yang, Z. Wu, B. Zhou, and S. Lin · 2021
Later among the works it cites.
Reinforcement learning with prototypical representations
D. Yarats, R. Fergus, A. Lazaric, and L. Pinto · 2021
Later among the works it cites.
Mastering atari games with limited data
W. Ye, S. Liu, T. Kurutach, P. Abbeel, and Y. Gao · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
C. Vondrick, A. Shrivastava, A. Fathi, S. Guadarrama, and K. Murphy · 2018
Cited alongside, same era.
Unsupervised feature learning via non-parametric instance discrimination
Z. Wu, Y. Xiong, S. X. Yu, and D. Lin · 2018
Cited alongside, same era.
mixup: Beyond empirical risk minimization
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz · 2018
Cited alongside, same era.
The sound of pixels
H. Zhao, C. Gan, A. Rouditchenko, C. Vondrick, J. McDermott, and A. Torralba · 2018
Cited alongside, same era.
Self-labelling via simultaneous clustering and representation learning
Y. M. Asano, C. Rupprecht, and A. Vedaldi · 2019
Cited alongside, same era.
Learning representations by maximizing mutual information across views
P. Bachman, R. D. Hjelm, and W. Buchwalter · 2019
Cited alongside, same era.
Mixmatch: A holistic approach to semi-supervised learning
D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel · 2019
Cited alongside, same era.
C.-H. Yeh, C.-Y. Hong, Y.-C. Hsu, T.-L. Liu, Y. Chen, and Y. LeCun · 2021
Later among the works it cites.
Florence: A New Foundation Model for Computer Vision
L. Yuan, D. Chen, Y.-L. Chen, N. Codella, X. Dai, J. Gao, H. Hu, X. Huang, B. Li, C. Li, C. Liu, M. Liu, Z. Liu, Y. Lu, Y. Shi, L. Wang, J. Wang, B. Xiao, Z. Xiao, J. Yang, M. Zeng, L. Zhou, and P. Zhang · 2021
Later among the works it cites.
Open-vocabulary object detection using captions
A. Zareian, K. D. Rosa, D. H. Hu, and S.-F. Chang · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny · 2021
Later among the works it cites.
Distilling localization for self-supervised representation learning
N. Zhao, Z. Wu, R. W. Lau, and S. Lin · 2021
Later among the works it cites.
α \alpha -req : Assessing representation quality in self-supervised learning by measuring eigenspectrum decay
K. K. Agrawal, A. K. Mondal, A. Ghosh, and B. A. Richards · 2022
Later among the works it cites.
Flamingo: A Visual Language Model for Few-Shot Learning
J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barreira, O. Vinyals, A. Zisserman, and K. Simonyan · 2022
Later among the works it cites.
On the Role of Bidirectionality in Language Model Pre-Training
M. Artetxe, J. Du, N. Goyal, L. Zettlemoyer, and V. Stoyanov · 2022
Later among the works it cites.
Holo-dex: Teaching dexterity with immersive mixed reality
S. P. Arunachalam, I. Güzey, S. Chintala, and L. Pinto · 2022
Later among the works it cites.
Masked siamese networks for label-efficient learning
M. Assran, M. Caron, I. Misra, P. Bojanowski, F. Bordes, P. Vincent, A. Joulin, M. Rabbat, and N. Ballas · 2022
Later among the works it cites.
Masked siamese networks for label-efficient learning
M. Assran, M. Caron, I. Misra, P. Bojanowski, F. Bordes, P. Vincent, A. Joulin, M. Rabbat, and N. Ballas · 2022
Later among the works it cites.
A. Baevski, A. Babu, W.-N. Hsu, and M. Auli · 2022
Later among the works it cites.
R. Balestriero and Y. LeCun · 2022
Later among the works it cites.
Detreg: Unsupervised pretraining with region priors for object detection
A. Bar, X. Wang, V. Kantorov, C. J. Reed, R. Herzig, G. Chechik, A. Rohrbach, T. Darrell, and A. Globerson · 2022
Later among the works it cites.
Vicregl: Variance-invariance-covariance regularization for self-supervised learning
A. Bardes, J. Ponce, and Y. LeCun · 2022
Later among the works it cites.
Maskgit: Masked generative image transformer
H. Chang, H. Zhang, L. Jiang, C. Liu, and W. T. Freeman · 2022
Later among the works it cites.
Clower: A pre-trained language model with contrastive learning over word and character representations
B. Chen, H. Tang, J. Bu, K. Zhang, J. Wang, Q. Wang, H.-T. Zheng, W. Wu, and L. Yu · 2022
Later among the works it cites.
Toward a geometrical understanding of self-supervised contrastive learning
R. Cosentino, A. Sengupta, S. Avestimehr, M. Soltanolkotabi, A. Ortega, T. Willke, and M. Tepper · 2022
Later among the works it cites.
From play to policy: Conditional behavior generation from uncurated robot data
Z. J. Cui, Y. Wang, N. M. M. Shafiullah, and L. Pinto · 2022
Later among the works it cites.
Flashattention: Fast and memory-efficient exact attention with io-awareness
T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Ré · 2022
Later among the works it cites.
An Empirical Study of Training End-to-End Vision-and-Language Transformers
Z.-Y. Dou, Y. Xu, Z. Gan, J. Wang, S. Wang, L. Wang, C. Zhu, P. Zhang, L. Yuan, N. Peng, Z. Liu, and M. Zeng · 2022
Later among the works it cites.
Improving Self-Supervised Learning by Characterizing Idealized Representations, Dec. 2022
Y. Dubois, T. Hashimoto, S. Ermon, and P. Liang · 2022
Later among the works it cites.
Contrastive learning as goal-conditioned reinforcement learning
B. Eysenbach, T. Zhang, R. Salakhutdinov, and S. Levine · 2022
Later among the works it cites.
Masked autoencoders as spatiotemporal learners
C. Feichtenhofer, H. Fan, Y. Li, and K. He · 2022
Later among the works it cites.
What do Vision Transformers Learn? A Visual Exploration
A. Ghiasi, H. Kazemi, E. Borgnia, S. Reich, M. Shu, M. Goldblum, A. G. Wilson, and T. Goldstein · 2022
Later among the works it cites.
Investigating power laws in deep representation learning
A. Ghosh, A. K. Mondal, K. K. Agrawal, and B. Richards · 2022
Later among the works it cites.
Omnimae: Single model masked pretraining on images and videos
R. Girdhar, A. El-Nouby, M. Singh, K. V. Alwala, A. Joulin, and I. Misra · 2022
Later among the works it cites.
Vision models are more robust and fair when pretrained on uncurated images without supervision
P. Goyal, Q. Duval, I. Seessel, M. Caron, M. Singh, I. Misra, L. Sagun, A. Joulin, and P. Bojanowski · 2022
Later among the works it cites.
Ego4d: Around the world in 3,000 hours of egocentric video
K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu, M. Martin, T. Nagarajan, I. Radosavovic, S. K. Ramakrishnan, F. Ryan, J. Sharma, M. Wray, M. Xu, E. Z. Xu, C. Zhao, S. Bansal, D. Batra, V. Cartillier, S. Crane, T. Do, M. Doulaty, A. Erapalli, C. Feichtenhofer, A. Fragomeni, Q. Fu, A. Gebreselasie, C. Gonzalez, J. Hillis, X. Huang, Y. Huang, W. Jia, W. Khoo, J. Kolar, S. Kottur, A. Kumar, F. Landini, C. Li, Y. Li, Z. Li, K. Mangalam, R. Modhugu, J. Munro, T. Murrell, T. Nishiyasu, W. Price, P. R. Puentes, M. Ramazanova, L. Sari, K. Somasundaram, A. Southerland, Y. Sugano, R. Tao, M. Vo, Y. Wang, X. Wu, T. Yagi, Z. Zhao, Y. Zhu, P. Arbelaez, D. Crandall, D. Damen, G. M. Farinella, C. Fuegen, B. Ghanem, V. K. Ithapu, C. V. Jawahar, H. Joo, K. Kitani, H. Li, R. Newcombe, A. Oliva, H. S. Park, J. M. Rehg, Y. Sato, J. Shi, M. Z. Shou, A. Torralba, L. Torresani, M. Yan, and J. Malik · 2022
Later among the works it cites.
Predictor networks and stop-grads provide implicit variance regularization in byol/simsiam
M. S. Halvagal, A. Laborieux, and F. Zenke · 2022
Later among the works it cites.
Exploring the gap between collapsed & whitened features in self-supervised learning
B. He and M. Ozay · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick · 2022
Later among the works it cites.
Robust self-supervised learning with lie groups
M. Ibrahim, D. Bouchacourt, and A. Morcos · 2022
Later among the works it cites.
D. Jarrett, C. Tallec, F. Altché, T. Mesnard, R. Munos, and M. Valko · 2022
Later among the works it cites.
Understanding dimensional collapse in contrastive self-supervised learning
L. Jing, P. Vincent, Y. LeCun, and Y. Tian · 2022
Later among the works it cites.
Self-supervised learning in medicine and healthcare
R. Krishnan, P. Rajpurkar, and E. J. Topol · 2022
Later among the works it cites.
xformers: A modular and hackable transformer modelling library
B. Lefaudeux, F. Massa, D. Liskovich, W. Xiong, V. Caggiano, S. Naren, M. Xu, J. Hu, M. Tintore, S. Zhang, P. Labatut, and D. Haziza · 2022
Later among the works it cites.
Vip: Towards universal visual reward and representation via value-implicit pre-training
Y. J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V. Kumar, and A. Zhang · 2022
Later among the works it cites.
Equivariant representation learning via class-pose decomposition
G. L. Marchetti, G. Tegnér, A. Varava, and D. Kragic · 2022
Later among the works it cites.
Variance-covariance regularization enforces pairwise independence in self-supervised representations
G. Mialon, R. Balestriero, and Y. Lecun · 2022
Later among the works it cites.
Simple open-vocabulary object detection with vision transformers
M. Minderer, A. Gritsenko, A. Stone, M. Neumann, D. Weissenborn, A. Dosovitskiy, A. Mahendran, A. Arnab, M. Dehghani, Z. Shen, et al · 2022
Later among the works it cites.
SLIP: Self-supervision Meets Language-Image Pre-training
N. Mu, A. Kirillov, D. Wagner, and S. Xie · 2022
Later among the works it cites.
R3m: A universal visual representation for robot manipulation
S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta · 2022
Later among the works it cites.
Expanding language-image pretrained models for general video recognition
B. Ni, H. Peng, M. Chen, S. Zhang, G. Meng, J. Fu, S. Xiang, and H. Ling · 2022
Later among the works it cites.
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen · 2022
Later among the works it cites.
The surprising effectiveness of representation learning for visual imitation
J. Pari, N. M. M. Shafiullah, S. P. Arunachalam, and L. Pinto · 2022
Later among the works it cites.
The unsurprising effectiveness of pre-trained vision models for control
S. Parisi, A. Rajeswaran, S. Purushwalkam, and A. Gupta · 2022
Later among the works it cites.
Learning symmetric embeddings for equivariant world models
J. Y. Park, O. Biza, L. Zhao, J. W. van de Meent, and R. Walters · 2022
Later among the works it cites.
Beit v2: Masked image modeling with vector-quantized visual tokenizers
Z. Peng, L. Dong, H. Bao, Q. Ye, and F. Wei · 2022
Later among the works it cites.
On the pros and cons of momentum encoder in self-supervised visual representation learning
T. Pham, C. Zhang, A. Niu, K. Zhang, and C. D. Yoo · 2022
Later among the works it cites.
A self-supervised descriptor for image copy detection, 2022
E. Pizzi, S. D. Roy, S. N. Ravindra, P. Goyal, and M. Douze · 2022
Later among the works it cites.
DenseCLIP: Language-Guided Dense Prediction With Context-Aware Prompting
Y. Rao, W. Zhao, G. Chen, Y. Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu · 2022
Later among the works it cites.
A Generalist Agent
S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-maron, M. Giménez, Y. Sulsky, J. Kay, J. T. Springenberg, T. Eccles, J. Bruce, A. Razavi, A. Edwards, N. Heess, Y. Chen, R. Hadsell, O. Vinyals, M. Bordbar, and N. de Freitas · 2022
Later among the works it cites.
Revisiting pretraining objectives for tabular deep learning
I. Rubachev, A. Alekberov, Y. Gorishniy, and A. Babenko · 2022
Later among the works it cites.
Improving the generalization of supervised models, 2022
M. B. Sariyildiz, Y. Kalantidis, K. Alahari, and D. Larlus · 2022
Later among the works it cites.
Self-supervised learning through efference copies
F. Scherr, Q. Guo, and T. Moraitis · 2022
Later among the works it cites.
LAION-5B: An open large-scale dataset for training next generation image-text models
C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, P. Schramowski, S. Kundurthy, K. Crowson, L. Schmidt, R. Kaczmarczyk, and J. Jitsev · 2022
Later among the works it cites.
How Much Can CLIP Benefit Vision-and-Language Tasks?
S. Shen, L. H. Li, H. Tan, M. Bansal, A. Rohrbach, K.-W. Chang, Z. Yao, and K. Keutzer · 2022
Later among the works it cites.
What do we maximize in self-supervised learning?
R. Shwartz-Ziv, R. Balestriero, and Y. LeCun · 2022
Later among the works it cites.
FLAVA: A Foundational Language and Vision Alignment Model
A. Singh, R. Hu, V. Goswami, G. Couairon, W. Galuba, M. Rohrbach, and D. Kiela · 2022
Later among the works it cites.
Joint embedding predictive architectures focus on slow features
V. Sobal, J. S V, S. Jalagam, N. Carion, K. Cho, and Y. LeCun · 2022
Later among the works it cites.
Unifying visual contrastive learning for object recognition from a graph perspective
S. Tang, F. Zhu, L. Bai, R. Zhao, C. Wang, and W. Ouyang · 2022
Later among the works it cites.
Unifying Language Learning Paradigms
Y. Tay, M. Dehghani, V. Q. Tran, X. Garcia, D. Bahri, T. Schuster, H. S. Zheng, N. Houlsby, and D. Metzler · 2022
Later among the works it cites.
Understanding deep contrastive learning via coordinate-wise optimization
Y. Tian · 2022
Later among the works it cites.
N. Tomasev, I. Bica, B. McWilliams, L. Buesing, R. Pascanu, C. Blundell, and J. Mitrovic · 2022
Later among the works it cites.
Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training
Z. Tong, Y. Song, J. Wang, and L. Wang · 2022
Later among the works it cites.
Deit iii: Revenge of the vit
H. Touvron, M. Cord, and H. Jégou · 2022
Later among the works it cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere, 2022
T. Wang and P. Isola · 2022
Later among the works it cites.
Contrastive learning rivals masked image modeling in fine-tuning via feature distillation
Y. Wei, H. Hu, Z. Xie, Z. Zhang, Y. Cao, J. Bao, D. Chen, and B. Guo · 2022
Later among the works it cites.
Mixing up contrastive learning: Self-supervised representation learning for time series
K. Wickstrøm, M. Kampffmeyer, K. Ø. Mikalsen, and R. Jenssen · 2022
Later among the works it cites.
Masked visual pre-training for motor control
T. Xiao, I. Radosavovic, T. Darrell, and J. Malik · 2022
Later among the works it cites.
Simmim: A simple framework for masked image modeling
Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, and H. Hu · 2022
Later among the works it cites.
Patch-level representation learning for self-supervised vision transformers
S. Yun, H. Lee, J. Kim, and J. Shin · 2022
Later among the works it cites.
Self-supervised learning of object parts for semantic segmentation
A. Ziegler and Y. M. Asano · 2022
Later among the works it cites.
Infonce is a variational autoencoder, 2023
L. Aitchison and S. Ganev · 2023
Closest in time.
Self-supervised learning from images with a joint-embedding predictive architecture, 2023
M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, and N. Ballas · 2023
Closest in time.
Muse: Text-to-image generation via masked generative transformers
H. Chang, H. Zhang, J. Barber, A. Maschinot, J. Lezama, L. Jiang, M.-H. Yang, K. Murphy, W. T. Freeman, M. Rubinstein, et al · 2023
Closest in time.
Self-supervised learning of split invariant equivariant representations
Q. Garrido, L. Najman, and Y. Lecun · 2023
Closest in time.
Dexterity from touch: Self-supervised pre-training of tactile representations with robotic play
I. Guzey, B. Evans, S. Chintala, and L. Pinto · 2023
Closest in time.
Searching large neighborhoods for integer linear programs with contrastive learning
T. Huang, A. Ferber, Y. Tian, B. Dilkina, and B. Steiner · 2023
Closest in time.
Transfer learning with deep tabular models
R. Levin, V. Cherepanova, A. Schwarzschild, A. Bansal, C. B. Bruss, T. Goldstein, A. G. Wilson, and M. Goldblum · 2023
Closest in time.
Dinov2: Learning robust visual features without supervision, 2023
M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, R. Howes, P.-Y. Huang, H. Xu, V. Sharma, S.-W. Li, W. Galuba, M. Rabbat, M. Assran, N. Ballas, G. Synnaeve, I. Misra, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski · 2023
Closest in time.
Understanding the role of nonlinearity in training dynamics of contrastive learning
Y. Tian · 2023
Closest in time.
Does zero-shot reinforcement learning exist?
A. Touati, J. Rapin, and Y. Ollivier · 2023
Closest in time.
Convnext v2: Co-designing and scaling convnets with masked autoencoders
S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie · 2023
Closest in time.
Doc: Improving long story coherence with detailed outline control
K. Yang, D. Klein, N. Peng, and Y. Tian · 2023
Closest in time.