Fetching the paper…
Reading the bibliography…
A common phenomena confining the representation quality in Self-Supervised Learning (SSL) is dimensional collapse (also known as rank degeneration), where the learned representations are mapped to a low dimensional subspace of the representation space.
Collective classification in network data
Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., and Eliassi-Rad, T. (2008) · 2008
Earlier work this paper cites.
Efficient estimation of word representations in vector space
Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013) · 2013
Earlier work this paper cites.
Glove: Global vectors for word representation
Pennington, J., Socher, R., and Manning, C. D. (2014) · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J. (2015a) · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J. (2015b) · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., and Dean, J. (2015) · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J. (2015) · 2015
Earlier work this paper cites.
Image-based recommendations on styles and substitutes
McAuley, J. J., Targett, C., Shi, Q., and van den Hengel, A. (2015) · 2015
Earlier work this paper cites.
An overview of microsoft academic service (mas) and applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. P., and Wang, K. (2015) · 2015
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Kipf, T. N. and Welling, M. (2017) · 2017
Earlier work this paper cites.
Predicting multicellular function through multi-layer tissue networks
Zitnik, M. and Leskovec, J. (2017) · 2017
Earlier work this paper cites.
Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking
Bojchevski, A. and Günnemann, S. (2018) · 2018
Earlier work this paper cites.
Learning word vectors for 157 languages
Grave, E., Bojanowski, P., Gupta, P., Joulin, A., and Mikolov, T. (2018) · 2018
Cited alongside, same era.
Deep graph infomax
Velickovic, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. (2018) · 2018
Cited alongside, same era.
Implicit regularization in deep matrix factorization
Arora, S., Cohen, N., Hu, W., and Luo, Y. (2019) · 2019
Cited alongside, same era.
Learning representations by maximizing mutual information across views
Bachman, P., Hjelm, R. D., and Buchwalter, W. (2019) · 2019
Cited alongside, same era.
Unsupervised deep learning by neighbourhood discovery
Huang, J., Dong, Q., Gong, S., and Zhu, X. (2019) · 2019
Cited alongside, same era.
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. E. (2020) · 2020
Cited alongside, same era.
Bootstrapped representation learning on graphs
Thakoor, S., Tallec, C., Azar, M. G., Munos, R., Velickovic, P., and Valko, M. (2021) · 2021
Later among the works it cites.
Understanding self-supervised learning dynamics without contrastive pairs
Tian, Y., Chen, X., and Ganguli, S. (2021) · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S. (2021) · 2021
Later among the works it cites.
Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods
Balestriero, R. and LeCun, Y. (2022) · 2022
Later among the works it cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Bardes, A., Ponce, J., and LeCun, Y. (2022) · 2022
Later among the works it cites.
Understanding dimensional collapse in contrastive self-supervised learning
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Deep graph contrastive representation learning
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. (2020) · 2020
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2021) · 2021
Cited alongside, same era.
Exploring simple siamese representation learning
Chen, X. and He, K. (2021) · 2021
Cited alongside, same era.
Whitening for self-supervised representation learning
Ermolov, A., Siarohin, A., Sangineto, E., and Sebe, N. (2021) · 2021
Cited alongside, same era.
On feature decorrelation in self-supervised learning
Hua, T., Wang, W., Xue, Z., Wang, Y., Ren, S., and Zhao, H. (2021) · 2021
Cited alongside, same era.
Graph barlow twins: A self-supervised representation learning framework for graphs
Bielak, P., Kajdanowicz, T., and Chawla, N. V. (2022a)
Cited in the paper.
Jing, L., Vincent, P., LeCun, Y., and Tian, Y. (2022) · 2022
Later among the works it cites.
Augmentation-free self-supervised learning on graphs
Lee, N., Lee, J., and Park, C. (2022) · 2022
Later among the works it cites.
Large-scale representation learning on graphs via bootstrapping
Thakoor, S., Tallec, C., Azar, M. G., Azabou, M., Dyer, E. L., Munos, R., Veličković, P., and Valko, M. (2022) · 2022
Later among the works it cites.
Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank
Garrido, Q., Balestriero, R., Najman, L., and Lecun, Y. (2023) · 2023
Later among the works it cites.
A critical look at the evaluation of GNNs under heterophily: Are we really making progress?
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. (2023) · 2023
Later among the works it cites.
Single-pass contrastive learning can work for both homophilic and heterophilic graph
Wang, H., Zhang, J., Zhu, Q., Huang, W., Kawaguchi, K., and Xiao, X. (2023) · 2023
Later among the works it cites.