Fetching the paper…
Reading the bibliography…
We study the relative effects of data augmentations, pretraining algorithms, and model architectures in Self-Supervised Learning (SSL).
Concerning nonnegative matrices and doubly stochastic matrices
Sinkhorn, R. and Knopp, P · 1967
Earlier work this paper cites.
A method of solving a convex programming problem with convergence rate o \ \backslash bigl(kˆ2 \ \backslash bigr)
Nesterov, Y. E · 1983
Earlier work this paper cites.
Improved baselines with momentum contrastive learning
Chen, X., Fan, H., Girshick, R., and He, K · 2003
Earlier work this paper cites.
Semi-supervised learning with graphs
Zhu, X · 2005
Earlier work this paper cites.
Active learning with real annotation costs
Settles, B., Craven, M., and Friedland, L · 2008
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Sutskever, I., Martens, J., Dahl, G., and Hinton, G · 2013
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Ioffe, S. and Szegedy, C · 2015
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L · 2015
Earlier work this paper cites.
{ \{ TensorFlow } \} : a system for { \{ Large-Scale } \} machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al · 2016
Earlier work this paper cites.
Gaussian error linear units (gelus)
Hendrycks, D. and Gimpel, K · 2016
Earlier work this paper cites.
Context encoders: Feature learning by inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A. A · 2016
Earlier work this paper cites.
Zagoruyko, S. and Komodakis, N · 2016
Earlier work this paper cites.
Colorful image colorization
Zhang, R., Isola, P., and Efros, A. A · 2016
Earlier work this paper cites.
Adam: A method for stochastic optimization, 2017
Kingma, D. P. and Ba, J · 2017
Earlier work this paper cites.
Large batch training of convolutional networks
You, Y., Gitman, I., and Ginsburg, B · 2017
Cited alongside, same era.
Compiling machine learning programs via high-level tracing
Frostig, R., Johnson, M. J., and Leary, C · 2018
Cited alongside, same era.
Unsupervised representation learning by predicting image rotations
Gidaris, S., Singh, P., and Komodakis, N · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O · 2018
Cited alongside, same era.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2019
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N · 2021
Later among the works it cites.
Want to reduce labeling cost? gpt-3 can help
Wang, S., Liu, Y., Xu, Y., Zhu, C., and Zeng, M · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S · 2021
Later among the works it cites.
Masked siamese networks for label-efficient learning
Assran, M., Caron, M., Misra, I., Bojanowski, P., Bordes, F., Vincent, P., Joulin, A., Rabbat, M., and Ballas, N · 2022
Later among the works it cites.
VICReg: Variance-invariance-covariance regularization for self-supervised learning
Bardes, A., Ponce, J., and LeCun, Y · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Pytorch: An imperative style, high-performance deep learning library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al · 2019
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A · 2020
Cited alongside, same era.
Exploring simple siamese representation learning. in 2021 ieee
Chen, X. and He, K · 2020
Cited alongside, same era.
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A · 2020
Cited alongside, same era.
Bootstrap your own latent-a new approach to self-supervised learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al · 2020
Cited alongside, same era.
Momentum contrast for unsupervised visual representation learning
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R · 2020
Cited alongside, same era.
Data-efficient image recognition with contrastive predictive coding
Henaff, O · 2020
Cited alongside, same era.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R · 2022
Later among the works it cites.
Weighted ensemble self-supervised learning
Ruan, Y., Singh, S., Morningstar, W. R., Alemi, A. A., Ioffe, S., Fischer, I., and Dillon, J. V · 2022
Later among the works it cites.
On the importance of asymmetry for siamese representation learning
Wang, X., Fan, H., Tian, Y., Kihara, D., and Chen, X · 2022
Later among the works it cites.
Self-supervised learning from images with a joint-embedding predictive architecture
Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., LeCun, Y., and Ballas, N · 2023
Later among the works it cites.
A cookbook of self-supervised learning
Balestriero, R., Ibrahim, M., Sobal, V., Morcos, A., Shekhar, S., Goldstein, T., Bordes, F., Bardes, A., Mialon, G., Tian, Y., et al · 2023
Later among the works it cites.
Kalibhat, N., Morningstar, W., Bijamov, A., Liu, L., Singhal, K., and Mansfield, P · 2023
Later among the works it cites.
Random field augmentations for self-supervised representation learning
Mansfield, P. A., Afkanpour, A., Morningstar, W. R., and Singhal, K · 2023
Later among the works it cites.
Dinov2: Learning robust visual features without supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al · 2023
Later among the works it cites.
Sassl: Enhancing self-supervised learning via neural style transfer
Rojas-Gomez, R. A., Singhal, K., Etemad, A., Bijamov, A., Morningstar, W. R., and Mansfield, P. A · 2023
Later among the works it cites.