Hyperspherically Regularized Networks for Self-Supervision

J. J. Thomson, Xxiv. on the structure of the atom: an investigation of the stability and periods of oscillation of a number of corpuscles arranged at equal intervals around the circumference of a circle; with application of the results to the theory of atomic structure, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 7 (39) (1904) 237–265

1904

Earlier work this paper cites.

X. Chen, H. Fan, R. Girshick, K. He, Improved baselines with momentum contrastive learning, arXiv preprint arXiv:2003.04297

Original

2003

Earlier work this paper cites.

S. Chopra, R. Hadsell, Y. LeCun, Learning a similarity metric discriminatively, with application to face verification, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1, IEEE, 2005, pp. 539–546

2005

Earlier work this paper cites.

Y. Tian, C. Sun, B. Poole, D. Krishnan, C. Schmid, P. Isola, What makes for good views for contrastive learning, arXiv preprint arXiv:2005.10243

Original

2005

Earlier work this paper cites.

P. H. Richemond, J.-B. Grill, F. Altché, C. Tallec, F. Strub, A. Brock, S. Smith, S. De, R. Pascanu, B. Piot, et al., Byol works even without batch statistics, arXiv preprint arXiv:2010.10241

Original

2010

Earlier work this paper cites.

T. Chen, L. Li, Intriguing properties of contrastive losses, arXiv preprint arXiv:2011.02803

Original

2011

Earlier work this paper cites.

H. Shi, D. Luo, S. Tang, J. Wang, Y. Zhuang, Run away from your teacher: Understanding byol by a novel self-supervised approach, arXiv preprint arXiv:2011.10944

Original

2011

Earlier work this paper cites.

S. Gidaris, A. Bursuc, G. Puy, N. Komodakis, M. Cord, P. Pérez, Online bag-of-visual-words generation for unsupervised representation learning, arXiv preprint arXiv:2012.11552

Original

2012

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

2016

Cited alongside, same era.

A. Tarvainen, H. Valpola, Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, in: Advances in neural information processing systems, 2017, pp. 1195–1204

2017

Cited alongside, same era.

A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Communications of the ACM 60 (6) (2017) 84–90

2017

Cited alongside, same era.

W. Liu, R. Lin, Z. Liu, L. Liu, Z. Yu, B. Dai, L. Song, Learning towards minimum hyperspherical energy, Advances in neural information processing systems 31 (2018) 6222–6233

2018

Cited alongside, same era.

M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, A. Joulin, Unsupervised learning of visual features by contrasting cluster assignments, in: 34th Conference on Neural Information Processing Systems, NeurIPS’20, Vol. 33, Curran Associates, Inc., 2020, pp. 9912–9924

2020

Later among the works it cites.

T. Wang, P. Isola, Understanding contrastive representation learning through alignment and uniformity on the hypersphere, in: International Conference on Machine Learning, PMLR, 2020, pp. 9929–9939

2020

Later among the works it cites.

K. He, H. Fan, Y. Wu, S. Xie, R. Girshick, Momentum contrast for unsupervised visual representation learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9729–9738

2020

Later among the works it cites.

A. Fetterman, J. Albrecht, Understanding self-supervised and contrastive learning with bootstrap your own latent (byol), https://www.untitled-ai.com/understanding-self-supervised-contrastive-learning.html (2020)

2020