Fetching the paper…
Reading the bibliography…
Joint-Embedding Self Supervised Learning (JE-SSL) has seen a rapid development, with the emergence of many method variations but only few principled guidelines that would help practitioners to successfully deploy them.
The approximation of one matrix by another of lower rank
Eckart, C. and Young, G · 1936
Earlier work this paper cites.
A mathematical theory of communication
Shannon, C. E · 1948
Earlier work this paper cites.
Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition
Cover, T. M · 1965
Earlier work this paper cites.
Signature verification using a “siamese” time delay neural network
Bromley, J., Guyon, I., LeCun, Y., Sackinger, E., and Shah, R · 1994
Earlier work this paper cites.
Improved baselines with momentum contrastive learning
Chen, X., Fan, H., Girshick, R., and He, K · 2003
Earlier work this paper cites.
The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A · 2007
Earlier work this paper cites.
Numerical recipes 3rd edition: The art of scientific computing
Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P · 2007
Earlier work this paper cites.
The effective rank: A measure of effective dimensionality
Roy, O. and Vetterli, M · 2007
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Sun database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A · 2010
Earlier work this paper cites.
3d object representations for fine-grained categorization
Krause, J., Stark, M., Deng, J., and Fei-Fei, L · 2013
Earlier work this paper cites.
Food-101 – mining discriminative components with random forests
Bossard, L., Guillaumin, M., and Van Gool, L · 2014
Earlier work this paper cites.
Learning deep features for scene recognition using places database
Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., and Oliva, A · 2014
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Accurate, large minibatch sgd: Training imagenet in 1 hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., and He, K · 2017
Earlier work this paper cites.
Clevr: A diagnostic dataset for compositional language and elementary visual reasoning
Johnson, J., Hariharan, B., Van Der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2017
Earlier work this paper cites.
Large batch training of convolutional networks
You, Y., Gitman, I., and Ginsburg, B · 2017
Cited alongside, same era.
Deep clustering for unsupervised learning
Caron, M., Bojanowski, P., Joulin, A., and Douze, M · 2018
Cited alongside, same era.
The inaturalist species classification and detection dataset
Horn, G. V., Aodha, O. M., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O · 2018
Cited alongside, same era.
How does batch normalization help optimization?
Santurkar, S., Tsipras, D., Ilyas, A., and Madry, A · 2018
Cited alongside, same era.
Deep image prior
An empirical study of training self-supervised vision transformers
Chen, X., Xie, S., and He, K · 2021
Later among the works it cites.
Whitening for self-supervised representation learning, 2021
Ermolov, A., Siarohin, A., Sangineto, E., and Sebe, N · 2021
Later among the works it cites.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T · 2021
Later among the works it cites.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R · 2021
Later among the works it cites.
On feature decorrelation in self-supervised learning
Hua, T., Wang, W., Xue, Z., Ren, S., Wang, Y., and Zhao, H · 2021
Later among the works it cites.
Compressive visual representations
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ulyanov, D., Vedaldi, A., and Lempitsky, V · 2018
Cited alongside, same era.
Unsupervised feature learning via non-parametric instance discrimination
Wu, Z., Xiong, Y., Yu, S., , and Lin, D · 2018
Cited alongside, same era.
Breaking the softmax bottleneck via learnable monotonic pointwise non-linearities
Ganea, O., Gelly, S., Bécigneul, G., and Severyn, A · 2019
Cited alongside, same era.
Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification
Helber, P., Bischke, B., Dengel, A., and Borth, D · 2019
Cited alongside, same era.
Local aggregation for unsupervised learning of visual embeddings
Zhuang, C., Zhai, A. L., and Yamins, D · 2019
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A · 2020
Cited alongside, same era.
Exploring simple siamese representation learning
Chen, X. and He, K · 2020
Cited alongside, same era.
Lee, K.-H., Arnab, A., Guadarrama, S., Canny, J., and Fischer, I · 2021
Later among the works it cites.
Selfaugment: Automatic augmentation policies for self-supervised learning
Reed, C. J., Metzger, S., Srinivas, A., Darrell, T., and Keutzer, K · 2021
Later among the works it cites.
Decoupled contrastive learning
Yeh, C.-H., Hong, C.-Y., Hsu, Y.-C., Liu, T.-L., Chen, Y., and LeCun, Y · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S · 2021
Later among the works it cites.
Balestriero, R. and LeCun, Y · 2022
Closest in time.
On the duality between contrastive and non-contrastive self-supervised learning
Garrido, Q., Chen, Y., Bardes, A., Najman, L., and Lecun, Y · 2022
Closest in time.
Investigating power laws in deep representation learning
Ghosh, A., Mondal, A. K., Agrawal, K. K., and Richards, B · 2022
Closest in time.
Girish, S., Dey, D., Joshi, N., Vineet, V., Shah, S., Mendes, C. C. T., Shrivastava, A., and Song, Y · 2022
Closest in time.
Exploring the gap between collapsed & whitened features in self-supervised learning
He, B. and Ozay, M · 2022
Closest in time.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R · 2022
Closest in time.
Understanding dimensional collapse in contrastive self-supervised learning
Jing, L., Vincent, P., LeCun, Y., and Tian, Y · 2022
Closest in time.
Tomasev, N., Bica, I., McWilliams, B., Buesing, L., Pascanu, R., Blundell, C., and Mitrovic, J · 2022
Closest in time.