Fetching the paper…
Reading the bibliography…
Self-supervised learning (SSL) pipelines differ in many design choices such as the architecture, augmentations, or pretraining data.
On uniform convergence of the frequencies of events to their probabilities
Vapnik, V. N. and Chervonenkis, A. Y · 1971
Earlier work this paper cites.
Neural networks and the bias/variance dilemma
Geman, S., Bienenstock, E., and Doursat, R · 1992
Earlier work this paper cites.
Approximation and estimation bounds for artificial neural networks
Barron, A. R · 1994
Earlier work this paper cites.
Bias plus variance decomposition for zero-one loss functions
Kohavi, R. and Wolpert, D. H · 1996
Earlier work this paper cites.
A unified bias-variance decomposition and its applications
Domingos, P · 2000
Earlier work this paper cites.
The Nature of Statistical Learning Theory
Vapnik, V. N · 2000
Earlier work this paper cites.
Improved baselines with momentum contrastive learning
Chen, X., Fan, H., Girshick, R., and He, K · 2003
Earlier work this paper cites.
Bias-variance analysis of support vector machines for the development of svm-based ensemble methods
Valentini, G. and Dietterich, T. G · 2004
Earlier work this paper cites.
Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization
Mukherjee, S., Niyogi, P., Poggio, T., and Rifkin, R · 2006
Earlier work this paper cites.
The tradeoffs of large scale learning
Bottou, L. and Bousquet, O · 2007
Earlier work this paper cites.
Algorithms for hyper-parameter optimization
Bergstra, J., Bardenet, R., Bengio, Y., and Kégl, B · 2011
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E · 2011
Earlier work this paper cites.
Understanding Machine Learning: From Theory to Algorithms
Shalev-Shwartz, S. and Ben-David, S · 2014
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
Doersch, C., Gupta, A., and Efros, A · 2015
Earlier work this paper cites.
In search of the real inductive bias: On the role of implicit regularization in deep learning
Neyshabur, B., Tomioka, R., and Srebro, N · 2015
Earlier work this paper cites.
Xgboost: A scalable tree boosting system
Chen, T. and Guestrin, C · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Unsupervised learning of visual representations by solving jigsaw puzzles
Noroozi, M. and Favaro, P · 2016
Earlier work this paper cites.
A unified approach to interpreting model predictions
Lundberg, S. M. and Lee, S · 2017
Earlier work this paper cites.
Deep clustering for unsupervised learning of visual features
Caron, M., Bojanowski, P., Joulin, A., and Douze, M · 2018
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
Gidaris, S., Singh, P., and Komodakis, N · 2018
Earlier work this paper cites.
A modern take on the bias-variance tradeoff in neural networks
Neal, B., Mittal, S., Baratin, A., Tantia, V., Scicluna, M., Lacoste-Julien, S., and Mitliagkas, I · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance discrimination
Wu, Z., Xiong, Y., Yu, S. X., and Lin, D · 2018
Earlier work this paper cites.
Learning representations by maximizing mutual information across views
Bachman, P., Hjelm, R. D., and Buchwalter, W · 2019
Earlier work this paper cites.
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., and Mandal, S · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M., Lee, K., and Toutanova, K · 2019
Earlier work this paper cites.
Scaling and benchmarking self-supervised visual representation learning
Goyal, P., Mahajan, D., Gupta, A., and Misra, I · 2019
Earlier work this paper cites.
On the bias-variance tradeoff: Textbooks need an update
Neal, B · 2019
Earlier work this paper cites.
Pytorch: An imperative style, high-performance deep learning library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S · 2019
Cited alongside, same era.
A theoretical analysis of contrastive unsupervised representation learning
Saunshi, N., Plevrakis, O., Arora, S., Khodak, M., and Khandeparkar, H · 2019
Cited alongside, same era.
Representation learning with contrastive predictive coding
van den Oord, A., Li, Y., and Vinyals, O · 2019
Cited alongside, same era.
Open problem: Monotonicity of learning
Viering, T., Mey, A., and Loog, M · 2019
Cited alongside, same era.
Towards good practices in self-supervised representation learning
Appalaraju, S., Zhu, Y., Xie, Y., and Fehérvári, I · 2020
Cited alongside, same era.
VISSL, 2021
Goyal, P., Duval, Q., Reizenstein, J., Leavitt, M., Xu, M., Lefaudeux, B., Singh, M., Reis, V., Caron, M., Bojanowski, P., Joulin, A., and Misra, I · 2021
Later among the works it cites.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T · 2021
Later among the works it cites.
On feature decorrelation in self-supervised learning
Hua, T., Wang, W., Xue, Z., Ren, S., Wang, Y., and Zhao, H · 2021
Later among the works it cites.
Representation learning via invariant causal mechanisms
Mitrovic, J., McWilliams, B., Walker, J., Buesing, L., and Blundell, C · 2021
Later among the works it cites.
MMSelfSup: Openmmlab self-supervised learning toolbox and benchmark
MMSelfSup · 2021
Later among the works it cites.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Self-labelling via simultaneous clustering and representation learning
Asano, Y. M., Rupprecht, C., and Vedaldi, A · 2020
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A · 2020
Cited alongside, same era.
Learning optimal representations with the decodable information bottleneck
Dubois, Y., Kiela, D., Schwab, D. J., and Vedantam, R · 2020
Cited alongside, same era.
Learning robust representations via multi-view information bottleneck
Federici, M., Dutta, A., Forré, P., Kushman, N., and Akata, Z · 2020
Cited alongside, same era.
Bootstrap Your Own Latent - a new approach to self-supervised learning
Grill, J. B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z., Azar, M. G., Piot, B., Kavukcuoglu, K., Munos, R., and Valko, M · 2020
Cited alongside, same era.
Momentum contrast for unsupervised visual representation learning
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R · 2020
Cited alongside, same era.
Scaling laws for neural language models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D · 2020
Cited alongside, same era.
Later among the works it cites.
Scaling laws for deep learning
Rosenfeld, J. S · 2021
Later among the works it cites.
Contrastive learning, multi-view redundancy, and linear models
Tosh, C., Krishnamurthy, A., and Hsu, D · 2021
Later among the works it cites.
Self-supervised learning from a multi-view perspective
Tsai, Y. H., Wu, Y., Salakhutdinov, R. R., and Morency, L · 2021
Later among the works it cites.
Understanding the behaviour of contrastive loss
Wang, F. and Liu, H · 2021
Later among the works it cites.
Dense contrastive learning for self-supervised visual pre-training
Wang, X., Zhang, R., Shen, C., Kong, T., and Li, L · 2021
Later among the works it cites.
On mutual information in contrastive learning for visual representations
Wu, M., Zhuang, C., Mosse, M., Yamins, D. L. K., and Goodman, N. D · 2021
Later among the works it cites.
Barlow Twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S · 2021
Later among the works it cites.
ibot: Image BERT pre-training with online tokenizer
Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A. L., and Kong, T · 2021
Later among the works it cites.
Masked siamese networks for label-efficient learning
Assran, M., Caron, M., Misra, I., Bojanowski, P., Bordes, F., Vincent, P., Joulin, A., R., M., and Ballas, N · 2022
Later among the works it cites.
Beit: BERT pre-training of image transformers
Bao, H., Dong, L., Piao, S., and Wei, F · 2022
Later among the works it cites.
Monotone learning
Bousquet, O. J., Daniely, A., Kaplan, H., Mansour, Y., Moran, S., and Stemmer, U · 2022
Later among the works it cites.
Reproducible scaling laws for contrastive language-image learning
Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., and Jitsev, J · 2022
Later among the works it cites.
Improving self-supervised learning by characterizing idealized representations
Dubois, Y., Hashimoto, T., Ermon, S., and Liang, P · 2022
Later among the works it cites.
Understanding and improving the role of projection head in self-supervised learning
Gupta, K., Ajanthan, T., Hengel, A. v. d., and Gould, S · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. B · 2022
Later among the works it cites.
Understanding dimensional collapse in contrastive self-supervised learning
Jing, L., Vincent, P., LeCun, Y., and Tian, Y · 2022
Later among the works it cites.
Instance-specific augmentation: Capturing local invariances
Miao, N., Mathieu, E., Dubois, Y., Rainforth, T., Teh, Y. W., Foster, A., and Kim, H · 2022
Later among the works it cites.
Optimal representations for covariate shift
Ruan, Y., Dubois, Y., and Maddison, C. J · 2022
Later among the works it cites.
Is a caption worth a thousand images? a controlled study for representation learning
Santurkar, S., Dubois, Y., Taori, R., Liang, P., and Hashimoto, T · 2022
Later among the works it cites.
Understanding contrastive learning requires incorporating inductive biases
Saunshi, N., Ash, J. T., Goel, S., Misra, D., Zhang, C., Arora, S., Kakade, S. M., and Krishnamurthy, A · 2022
Later among the works it cites.
Chaos is a ladder: A new theoretical understanding of contrastive learning via augmentation overlap
Wang, Y., Zhang, Y., Wang, Y., Yang, J., and Lin, Z · 2022
Later among the works it cites.
Beit v2: Masked image modeling with vector-quantized visual tokenizers
Zhiliang, P., Li, D., Bao, H., Ye, Q., and Wei, F · 2022
Later among the works it cites.