Fetching the paper…
Reading the bibliography…
Prevention of complete and dimensional collapse of representations has recently become a design principle for self-supervised learning (SSL).
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Hsu, H., Qi, H., and Brown, M. (2019) · 1909
Earlier work this paper cites.
Theoretical neuroscience: computational and mathematical modeling of neural systems
Dayan, P. and Abbott, L. F. (2005) · 2005
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · 2010
Earlier work this paper cites.
Representation learning via invariant causal mechanisms
Mitrovic, J., McWilliams, B., Walker, J., Buesing, L., and Blundell, C. (2020) · 2010
Earlier work this paper cites.
Statistical Physics: Volume 5
Landau, L. D. and Lifshitz, E. M. (2013) · 2013
Earlier work this paper cites.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O. (2018) · 2018
Earlier work this paper cites.
A Theoretical Analysis of Contrastive Unsupervised Representation Learning
Arora, S., Khandeparkar, H., Khodak, M., Plevrakis, O., and Saunshi, N. (2019) · 2019
Earlier work this paper cites.
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2020) · 2020
Earlier work this paper cites.
Bootstrap your own latent: A new approach to self-supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., Piot, B., kavukcuoglu, k., Munos, R., and Valko, M. (2020) · 2020
Earlier work this paper cites.
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., and Girschick, R. (2020) · 2020
Earlier work this paper cites.
Prevalence of neural collapse during the terminal phase of deep learning training
Papyan, V., Han, X., and Donoho, D. L. (2020) · 2020
Earlier work this paper cites.
What makes for good views for contrastive learning?
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., and Isola, P. (2020) · 2020
Earlier work this paper cites.
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
Wang, T. and Isola, P. (2020) · 2020
Earlier work this paper cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Bardes, A., Ponce, J., and LeCun, Y. (2021) · 2021
Cited alongside, same era.
Emerging Properties in Self-Supervised Vision Transformer
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021) · 2021
Cited alongside, same era.
Exploring Simple Siamese Representation Learning
Chen, X. and He, K. (2021) · 2021
Cited alongside, same era.
Whitening for self-supervised representation learning
Ermolov, A., Siarohin, A., Sangineto, E., and Sebe, N. (2021) · 2021
Cited alongside, same era.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T. (2021) · 2021
Cited alongside, same era.
Understanding the behaviour of contrastive loss
Wang, F. and Liu, H. (2021) · 2021
Later among the works it cites.
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
Wei, C., Shen, K., Chen, Y., and Ma, T. (2021) · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S. (2021) · 2021
Later among the works it cites.
Contrastive Learning Inverts the Data Generating Process
Zimmerman, R., Sharma, Y., Schneider, S., Bethge, M., and Brendel, W. (2021) · 2021
Later among the works it cites.
Toward a geometrical understanding of self-supervised contrastive learning
Cosentino, R., Sengupta, A., Avestimehr, S., Soltanolkotabi, M., Ortega, A., Willke, T., and Tepper, M. (2022) · 2022
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
On feature decorrelation in self-supervised learning
Hua, T., Wang, W., Xue, Z., Ren, S., Wang, Y., and Zhao, H. (2021) · 2021
Cited alongside, same era.
Understanding dimensional collapse in contrastive self-supervised learning
Jing, L., Vincent, P., LeCun, Y., and Tian, Y. (2021) · 2021
Cited alongside, same era.
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style
Kugelgen, J., Sharma, Y., Gresle, L., Brendel, W., Scholkopf, B., Besserve, M., and Locatello, F. (2021) · 2021
Cited alongside, same era.
Self-supervised learning is more robust to dataset imbalance
Liu, H., HaoChen, J. Z., Gaidon, A., and Ma, T. (2021) · 2021
Cited alongside, same era.
Understanding negative samples in instance discriminative self-supervised representation learning
Nozawa, K. and Sato, I. (2021) · 2021
Cited alongside, same era.
Can contrastive learning avoid shortcut solutions?
Robinson, J., Sun, L., Yu, K., Batmanghelich, K., Jegelka, S., and Sra, S. (2021) · 2021
Cited alongside, same era.
Understanding self-supervised Learning Dynamics without Contrastive Pairs
Tian, Y., Chen, X., and Ganguli, S. (2021) · 2021
Cited alongside, same era.
Pokle, A., Tian, J., Li, Y., and Risteski, A. (2022) · 2022
Closest in time.
Understanding Contrastive Learning Requires Incorporating Inductive Biases
Saunshi, N., Ash, J., Goel, S., Misra, D., Zhang, C., Arora, S., Kakade, S., and Krishnamurthy, A. (2022) · 2022
Closest in time.
Deep contrastive learning is provably (almost) principal component analysis
Tian, Y. (2022) · 2022
Closest in time.
Analyzing data-centric properties for contrastive learning on graphs
Trivedi, P., Lubana, E. S., Heimann, M., Koutra, D., and Thiagarajan, J. J. (2022) · 2022
Closest in time.
Chaos is a ladder: A new theoretical understanding of contrastive learning via augmentation overlap
Wang, Y., Zhang, Q., Wang, Y., Yang, J., and Lin, Z. (2022) · 2022
Closest in time.
Posterior collapse of a linear latent variable model
Wang, Z. and Ziyin, L. (2022) · 2022
Closest in time.
Exact phase transitions in deep learning
Ziyin, L. and Ueda, M. (2022) · 2022
Closest in time.