Fetching the paper…
Reading the bibliography…
Exponential Moving Average (EMA or momentum) is widely used in modern self-supervised learning (SSL) approaches, such as MoCo, for enhancing performance.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in neural information processing systems 33 (2020) 1877–1901
1901
Earlier work this paper cites.
M. J. Pring, Technical analysis explained: The successful investor’s guide to spotting investment trends and turning points, McGraw-Hill Professional, 2002
2002
Earlier work this paper cites.
G. Appel, Technical analysis: power tools for active investors, FT Press, 2005
2005
Earlier work this paper cites.
A. Krizhevsky, et al., Learning multiple layers of features from tiny images, . (2009)
2009
Earlier work this paper cites.
F. Klinker, Exponential moving average versus moving exponential average, Mathematische Semesterberichte 58 (1) (2011) 97–107
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, 2012
2012
Earlier work this paper cites.
I. Sutskever, J. Martens, G. Dahl, G. Hinton, On the importance of initialization and momentum in deep learning, in: International conference on machine learning, PMLR, 2013, pp. 1139–1147
2013
Earlier work this paper cites.
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: International Conference on Learning Representations, 2015
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016
2016
Earlier work this paper cites.
A. Tarvainen, H. Valpola, Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, Advances in neural information processing systems 30 (2017)
2017
Earlier work this paper cites.
N. Komodakis, S. Gidaris, Unsupervised representation learning by predicting image rotations, in: International Conference on Learning Representations (ICLR), 2018
2018
Earlier work this paper cites.
2018
Earlier work this paper cites.
S. Santurkar, D. Tsipras, A. Ilyas, A. Madry, How does batch normalization help optimization?, in: NeurIPS, 2018
2018
Earlier work this paper cites.
J. Ma, D. Yarats, Quasi-hyperbolic momentum and adam for deep learning, in: International Conference on Learning Representations, 2018
2018
Earlier work this paper cites.
T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, in: International conference on machine learning, PMLR, 2018, pp. 1861–1870
2018
Earlier work this paper cites.
A. Iscen, G. Tolias, Y. Avrithis, O. Chum, Mining on manifolds: Metric learning without labels, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7642–7651
2018
Earlier work this paper cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, Language models are unsupervised multitask learners, OpenAI blog (2019)
2019
Earlier work this paper cites.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, Pytorch: An imperative style, high-performance deep learning library, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019, pp. 8024–8035
2019
Earlier work this paper cites.
Y. Tian, D. Krishnan, P. Isola, Contrastive representation distillation, in: International Conference on Learning Representations, 2019
2019
Earlier work this paper cites.
Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, R. Soricut, Albert: A lite bert for self-supervised learning of language representations, in: International Conference on Learning Representations, 2020
2020
Earlier work this paper cites.
W. Su, X. Zhu, Y. Cao, B. Li, L. Lu, F. Wei, J. Dai, {VL}-{bert}: Pre-training of generic visual-linguistic representations, in: International Conference on Learning Representations, 2020
2020
Earlier work this paper cites.
P. Nie, Y. Zhang, X. Geng, A. Ramamurthy, L. Song, D. Jiang, Dc-bert: Decoupling question and document for efficient contextual encoding, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 1829–1832
2020
Earlier work this paper cites.
K. He, H. Fan, Y. Wu, S. Xie, R. Girshick, Momentum contrast for unsupervised visual representation learning, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
2020
Earlier work this paper cites.
T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: International Conference on Machine Learning, 2020
2020
Cited alongside, same era.
2020
Cited alongside, same era.
J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, et al., Bootstrap your own latent-a new approach to self-supervised learning, Advances in Neural Information Processing Systems 33 (2020) 21271–21284
2020
Cited alongside, same era.
M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, A. Joulin, Unsupervised learning of visual features by contrasting cluster assignments, Advances in Neural Information Processing Systems 33 (2020) 9912–9924
2020
Cited alongside, same era.
2021
Later among the works it cites.
J. Von Kügelgen, Y. Sharma, L. Gresele, W. Brendel, B. Schölkopf, M. Besserve, F. Locatello, Self-supervised learning with data augmentations provably isolates content from style, Advances in Neural Information Processing Systems 34 (2021)
2021
Later among the works it cites.
H. Wu, Y. Qu, S. Lin, J. Zhou, R. Qiao, Z. Zhang, Y. Xie, L. Ma, Contrastive learning for compact single image dehazing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 10551–10560
2021
Later among the works it cites.
T. Pan, Y. Song, T. Yang, W. Jiang, W. Liu, Videomoco: Contrastive video representation learning with temporally adversarial examples, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 11205–11214
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Y. Tian, D. Krishnan, P. Isola, Contrastive multiview coding, in: Proceedings of the European conference on computer vision (ECCV), Springer, 2020, pp. 776–794
2020
Cited alongside, same era.
N. Vieillard, B. Scherrer, O. Pietquin, M. Geist, Momentum in reinforcement learning, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2020, pp. 2529–2538
2020
Cited alongside, same era.
E. Korkmaz, Nesterov momentum adversarial perturbations in the deep reinforcement learning domain, in: International Conference on Machine Learning, ICML 2020, Inductive Biases, Invariances and Generalization in Reinforcement Learning Workshop, Vol. 1, 2020
2020
Cited alongside, same era.
T. Chen, S. Kornblith, K. Swersky, M. Norouzi, G. E. Hinton, Big self-supervised models are strong semi-supervised learners, Advances in neural information processing systems 33 (2020) 22243–22255
2020
Cited alongside, same era.
doi:https://doi.org/10.1016/j.patcog.2020.107535
T. M. Paixão, R. F. Berriel, M. C. Boeres, A. L. Koerich, C. Badue, A. F. De Souza, T. Oliveira-Santos, Self-supervised deep reconstruction of mixed strip-shredded text documents, Pattern Recognition 107 (2020) 107535 · 2020
Cited alongside, same era.
H. Eun, J. Moon, J. Park, C. Jung, C. Kim, Learning to discriminate information for online action detection, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
2020
Cited alongside, same era.
C. Zhuang, T. She, A. Andonian, M. S. Mark, D. Yamins, Unsupervised learning from video with deep neural embeddings, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
2020
Cited alongside, same era.
K. Zeng, M. Ning, Y. Wang, Y. Guo, Hierarchical clustering with hard-batch triplet loss for person re-identification, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
2020
Cited alongside, same era.
2021
Later among the works it cites.
F. Wang, H. Liu, Understanding the behaviour of contrastive loss, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 2495–2504
2021
Later among the works it cites.
H. Hu, J. Cui, L. Wang, Region-aware contrastive learning for semantic segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 16291–16301
2021
Later among the works it cites.
A. Diba, V. Sharma, R. Safdari, D. Lotfi, S. Sarfraz, R. Stiefelhagen, L. Van Gool, Vi2clr: Video and image for visual contrastive learning of representation, in: ICCV, 2021
2021
Later among the works it cites.
2021
Later among the works it cites.
J. D. Robinson, C.-Y. Chuang, S. Sra, S. Jegelka, Contrastive learning with hard negative samples, in: International Conference on Learning Representations, 2021
2021
Later among the works it cites.
A. Kaku, S. Upadhya, N. Razavian, Intermediate layers matter in momentum contrastive self supervised learning, Advances in Neural Information Processing Systems 34 (2021)
2021
Later among the works it cites.
X. Wang, R. Zhang, C. Shen, T. Kong, L. Li, Dense contrastive learning for self-supervised visual pre-training, in: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2021
2021
Later among the works it cites.
Y. Tian, O. J. Henaff, A. van den Oord, Divide and contrast: Self-supervised learning from uncurated data, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10063–10074
2021
Later among the works it cites.
A. Ermolov, A. Siarohin, E. Sangineto, N. Sebe, Whitening for self-supervised representation learning, in: International Conference on Machine Learning, PMLR, 2021
2021
Later among the works it cites.
J. Zbontar, L. Jing, I. Misra, Y. LeCun, S. Deny, Barlow twins: Self-supervised learning via redundancy reduction, International Conference on Machine Learning (2021)
2021
Later among the works it cites.
doi:https://doi.org/10.1016/j.patcog.2022.108767
J. Liu, Z. Qi, B. Wang, Y. Tian, Y. Shi, Self-llp: Self-supervised learning from label proportions with self-ensemble, Pattern Recognition 129 (2022) 108767 · 2022
Closest in time.
D. Madaan, J. Yoon, Y. Li, Y. Liu, S. J. Hwang, Representational continuity for unsupervised continual learning , in: International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=9Hrka5PA7LW
2022
Closest in time.
C. Zhang, K. Zhang, C. Zhang, T. X. Pham, C. D. Yoo, I. S. Kweon, How does simsiam avoid collapse without negative samples? a unified understanding with self-supervised contrastive learning, in: International Conference on Learning Representations, 2022
2022
Closest in time.
2022
Closest in time.
V. G. T. da Costa, E. Fini, M. Nabi, N. Sebe, E. Ricci, solo-learn: A library of self-supervised methods for visual representation learning , Journal of Machine Learning Research 23 (56) (2022) 1–6. URL http://jmlr.org/papers/v23/21-1155.html
2022
Closest in time.
K. Lee, Prototypical contrastive predictive coding , in: International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=8la28hZOwug
2022
Closest in time.
2022
Closest in time.
doi:https://doi.org/10.1016/j.patcog.2022.108784
Z. Zhang, J. Sun, Y. Dai, D. Zhou, X. Song, M. He, Self-supervised rigid transformation equivariance for accurate 3d point cloud registration, Pattern Recognition 130 (2022) 108784 · 2022
Closest in time.