Fetching the paper…
Reading the bibliography…
In this paper, we conduct a comprehensive analysis of two dual-branch (Siamese architecture) self-supervised learning approaches, namely Barlow Twins and spectral contrastive learning, through the lens of matrix mutual information.
The effective rank: A measure of effective dimensionality
Roy, O. and Vetterli, M · 2007
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Mathematische grundlagen der quantenmechanik , volume 38
Von Neumann, J · 2013
Earlier work this paper cites.
Measures of entropy from data using infinitely divisible kernels
Giraldo, L. G. S., Rao, M., and Principe, J. C · 2014
Earlier work this paper cites.
Tutorial on variational autoencoders
Doersch, C · 2016
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y · 2018
Earlier work this paper cites.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance discrimination
Wu, Z., Xiong, Y., Yu, S. X., and Lin, D · 2018
Earlier work this paper cites.
A theoretical analysis of contrastive unsupervised representation learning
Arora, S., Khandeparkar, H., Khodak, M., Plevrakis, O., and Saunshi, N · 2019
Earlier work this paper cites.
Learning representations by maximizing mutual information across views
Bachman, P., Hjelm, R. D., and Buchwalter, W · 2019
Earlier work this paper cites.
Tian, Y., Krishnan, D., and Isola, P · 2019
Earlier work this paper cites.
Unsupervised embedding learning via invariant and spreading instance feature
Ye, M., Zhang, X., Yuen, P. C., and Chang, S.-F · 2019
Earlier work this paper cites.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A · 2020
Earlier work this paper cites.
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G · 2020
Earlier work this paper cites.
Bootstrap your own latent-a new approach to self-supervised learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al · 2020
Earlier work this paper cites.
Data-efficient image recognition with contrastive predictive coding
Henaff, O · 2020
Earlier work this paper cites.
Predicting what you already know helps: Provable self-supervised learning
Lee, J. D., Lei, Q., Saunshi, N., and Zhuo, J · 2020
Earlier work this paper cites.
Self-supervised learning of pretext-invariant representations
Misra, I. and Maaten, L. v. d · 2020
Earlier work this paper cites.
What makes for good views for contrastive learning
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., and Isola, P · 2020
Earlier work this paper cites.
Contrastive estimation reveals topic posterior information to linear models
Tosh, C., Krishnamurthy, A., and Hsu, D · 2020
Earlier work this paper cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Wang, T. and Isola, P · 2020
Cited alongside, same era.
Learning diverse and discriminative representations via the principle of maximal coding rate reduction
Yu, Y., Chan, K. H. R., You, C., Song, C., and Ma, Y · 2020
Cited alongside, same era.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Bardes, A., Ponce, J., and LeCun, Y · 2021
Cited alongside, same era.
Emerging properties in self-supervised vision transformers
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A · 2021
Cited alongside, same era.
Exploring simple siamese representation learning
Chen, X. and He, K · 2021
Cited alongside, same era.
How to understand masked autoencoders
Cao, S., Xu, P., and Clifton, D. A · 2022
Later among the works it cites.
Improving self-supervised learning by characterizing idealized representations
Dubois, Y., Hashimoto, T., Ermon, S., and Liang, P · 2022
Later among the works it cites.
On the duality between contrastive and non-contrastive self-supervised learning
Garrido, Q., Chen, Y., Bardes, A., Najman, L., and Lecun, Y · 2022
Later among the works it cites.
Beyond separability: Analyzing the linear transferability of contrastive representations to related subpopulations
HaoChen, J. Z., Wei, C., Kumar, A., and Ma, T · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Gao, T., Yao, X., and Chen, D · 2021
Cited alongside, same era.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T · 2021
Cited alongside, same era.
Towards the generalization of contrastive self-supervised learning
Huang, W., Yi, M., and Zhao, X · 2021
Cited alongside, same era.
Predicting what you already know helps: Provable self-supervised learning
Lee, J. D., Lei, Q., Saunshi, N., and Zhuo, J · 2021
Cited alongside, same era.
Self-supervised learning with kernel dependence maximization
Li, Y., Pogodin, R., Sutherland, D. J., and Gretton, A · 2021
Cited alongside, same era.
Understanding negative samples in instance discriminative self-supervised representation learning
Nozawa, K. and Sato, I · 2021
Cited alongside, same era.
Contrastive learning with hard negative samples
Robinson, J. D., Chuang, C.-Y., Sra, S., and Jegelka, S · 2021
Cited alongside, same era.
Later among the works it cites.
Your contrastive learning is secretly doing stochastic neighbor embedding
Hu, T., Liu, Z., Zhou, F., Wang, W., and Huang, W · 2022
Later among the works it cites.
Self-supervised learning via maximum entropy coding
Liu, X., Wang, Z., Li, Y.-L., and Wang, S · 2022
Later among the works it cites.
Contrasting the landscape of contrastive and non-contrastive learning
Pokle, A., Tian, J., Li, Y., and Risteski, A · 2022
Later among the works it cites.
Exploring the equivalence of siamese self-supervised learning via a unified gradient framework
Tao, C., Wang, H., Zhu, X., Dong, J., Song, S., Huang, G., and Dai, J · 2022
Later among the works it cites.
Deep contrastive learning is provably (almost) principal component analysis
Tian, Y · 2022
Later among the works it cites.
Chaos is a ladder: A new theoretical understanding of contrastive learning via augmentation overlap
Wang, Y., Zhang, Q., Wang, Y., Yang, J., and Lin, Z · 2022
Later among the works it cites.
The mechanism of prediction head in non-contrastive self-supervised learning
Wen, Z. and Li, Y · 2022
Later among the works it cites.
Simmim: A simple framework for masked image modeling
Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., and Hu, H · 2022
Later among the works it cites.
Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank
Garrido, Q., Balestriero, R., Najman, L., and Lecun, Y · 2023
Closest in time.
The representation jensen-shannon divergence
Hoyos-Osorio, J. K. and Sanchez-Giraldo, L. G · 2023
Closest in time.
Understanding masked autoencoders via hierarchical latent variable models
Kong, L., Ma, M. Q., Chen, G., Xing, E. P., Chi, Y., Morency, L.-P., and Zhang, K · 2023
Closest in time.
To compress or not to compress–self-supervised learning and information theory: A review
Shwartz-Ziv, R. and LeCun, Y · 2023
Closest in time.
An information-theoretic perspective on variance-invariance-covariance regularization
Shwartz-Ziv, R., Balestriero, R., Kawaguchi, K., Rudner, T. G., and LeCun, Y · 2023
Closest in time.
Dime: Maximizing mutual information by a difference of matrix-based entropies
Skean, O., Osorio, J. K. H., Brockmeier, A. J., and Giraldo, L. G. S · 2023
Closest in time.
Contrastive learning is spectral clustering on similarity graph
Tan, Z., Zhang, Y., Yang, J., and Yuan, Y · 2023
Closest in time.