Fetching the paper…
Reading the bibliography…
Non-contrastive methods of self-supervised learning (such as BYOL and SimSiam) learn representations by minimizing the distance between two views of the same image.
On the perturbation of pseudo-inverses, projections and linear least squares problems
Stewart, G. W · 1977
Earlier work this paper cites.
Signature verification using a“ siamese” time delay neural network
Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., and Shah, R · 1994
Earlier work this paper cites.
What makes for good views for contrastive learning?
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., and Isola, P · 2005
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Understanding self-supervised learning with dual deep networks
Tian, Y., Yu, L., Chen, X., and Ganguli, S · 2010
Earlier work this paper cites.
Introduction to the non-asymptotic analysis of random matrices
Vershynin, R · 2010
Earlier work this paper cites.
An analysis of single-layer networks in unsupervised feature learning
Coates, A., Ng, A., and Lee, H · 2011
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Saxe, A. M., McClelland, J. L., and Ganguli, S · 2013
Earlier work this paper cites.
Learning mixtures of gaussians in high dimensions
Ge, R., Huang, Q., and Kakade, S. M · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Deep learning without poor local minima
Kawaguchi, K · 2016
Earlier work this paper cites.
Barlow twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S · 2016
Earlier work this paper cites.
Large batch training of convolutional networks
You, Y., Gitman, I., and Ginsburg, B · 2017
Earlier work this paper cites.
On the optimization of deep networks: Implicit acceleration by overparameterization
Arora, S., Cohen, N., and Hazan, E · 2018
Cited alongside, same era.
An analytic theory of generalization dynamics and transfer learning in deep linear networks
Lampinen, A. K. and Ganguli, S · 2018
Cited alongside, same era.
Deep linear networks with arbitrary loss: All local minima are global
Laurent, T. and Brecht, J · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O · 2018
Cited alongside, same era.
High-dimensional probability: An introduction with applications in data science , volume 47
Vershynin, R · 2018
Cited alongside, same era.
Bootstrap your own latent: A new approach to self-supervised learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al · 2020
Later among the works it cites.
Momentum contrast for unsupervised visual representation learning
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R · 2020
Later among the works it cites.
Predicting what you already know helps: Provable self-supervised learning
Lee, J. D., Lei, Q., Saunshi, N., and Zhuo, J · 2020
Later among the works it cites.
Contrastive learning, multi-view redundancy, and linear models
Tosh, C., Krishnamurthy, A., and Hsu, D · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Arora, S., Cohen, N., Golowich, N., and Hu, W · 2019
Cited alongside, same era.
Learning representations by maximizing mutual information across views
Bachman, P., Hjelm, R. D., and Buchwalter, W · 2019
Cited alongside, same era.
Width provably matters in optimization for deep linear neural networks
Du, S. and Hu, W · 2019
Cited alongside, same era.
A mathematical theory of semantic development in deep neural networks
Saxe, A. M., McClelland, J. L., and Ganguli, S · 2019
Cited alongside, same era.
Tian, Y., Krishnan, D., and Isola, P · 2019
Cited alongside, same era.
High-dimensional statistics: A non-asymptotic viewpoint , volume 48
Wainwright, M. J · 2019
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A · 2020
Cited alongside, same era.
Tsai, Y.-H. H., Wu, Y., Salakhutdinov, R., and Morency, L.-P · 2020
Later among the works it cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Wang, T. and Isola, P · 2020
Later among the works it cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Bardes, A., Ponce, J., and LeCun, Y · 2021
Closest in time.
Emerging properties in self-supervised vision transformers
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A · 2021
Closest in time.
Whitening for self-supervised representation learning
Ermolov, A., Siarohin, A., Sangineto, E., and Sebe, N · 2021
Closest in time.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T · 2021
Closest in time.
On feature decorrelation in self-supervised learning
Hua, T., Wang, W., Xue, Z., Wang, Y., Ren, S., and Zhao, H · 2021
Closest in time.
Understanding self-supervised learning dynamics without contrastive pairs
Tian, Y., Chen, X., and Ganguli, S · 2021
Closest in time.
Contrastive learning, multi-view redundancy, and linear models
Tosh, C., Krishnamurthy, A., and Hsu, D · 2021
Closest in time.
Toward understanding the feature learning process of self-supervised contrastive learning
Wen, Z. and Li, Y · 2021
Closest in time.