Fetching the paper…
Reading the bibliography…
Representations learned via self-supervised learning (SSL) can be susceptible to dimensional collapse, where the learned representation subspace is of extremely low dimensionality and thus fails to represent the full data distribution and modalities.
An intrinsic dimensionality estimator from near-neighbor information
Karl W Pettis, Thomas A Bailey, Anil K Jain, and Richard C Dubes · 1979
Earlier work this paper cites.
How should relative changes be measured?
Leo Törnqvist, Pentti Vartia, and Yrjö O Vartia · 1985
Earlier work this paper cites.
Autoencoders, minimum description length and helmholtz free energy
Geoffrey E Hinton and Richard Zemel · 1993
Earlier work this paper cites.
Intrinsic dimensionality estimation with optimally topology preserving maps
Jörg Bruske and Gerald Sommer · 1998
Earlier work this paper cites.
Maximum likelihood estimation of intrinsic dimension
Elizaveta Levina and Peter Bickel · 2004
Earlier work this paper cites.
Learning on statistical manifolds for clustering and visualization
Kevin M Carter, Raviv Raich, and AO Hero · 2007
Earlier work this paper cites.
The effective rank: A measure of effective dimensionality
Olivier Roy and Martin Vetterli · 2007
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey Hinton · 2009
Earlier work this paper cites.
Dimensional testing for multi-step similarity search
Michael E Houle, Xiguo Ma, Michael Nett, and Vincent Oria · 2012
Earlier work this paper cites.
Collecting a large-scale dataset of fine-grained cars
Jonathan Krause, Jia Deng, Michael Stark, and Li Fei-Fei · 2013
Earlier work this paper cites.
Birdsnap: Large-scale fine-grained visual categorization of birds
Thomas Berg, Jiongxin Liu, Seung Woo Lee, Michelle L Alexander, David W Jacobs, and Peter N Belhumeur · 2014
Earlier work this paper cites.
Food-101–mining discriminative components with random forests
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool · 2014
Earlier work this paper cites.
Danco: An intrinsic dimensionality estimator exploiting angle and norm concentration
Claudio Ceruti, Simone Bassis, Alessandro Rozza, Gabriele Lombardi, Elena Casiraghi, and Paola Campadelli · 2014
Earlier work this paper cites.
Describing textures in the wild
Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi · 2014
Earlier work this paper cites.
Rich feature hierarchies for accurate object detection and semantic segmentation
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik · 2014
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick · 2014
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts
Ilya Loshchilov and Frank Hutter · 2016
Earlier work this paper cites.
Measuring dependency via intrinsic dimensionality
Simone Romano, Oussama Chelly, Vinh Nguyen, James Bailey, and Michael E Houle · 2016
Earlier work this paper cites.
Large batch training of convolutional networks
Yang You, Igor Gitman, and Boris Ginsburg · 2017
Earlier work this paper cites.
Extreme-value-theoretic estimation of local intrinsic dimensionality
Laurent Amsaleg, Oussama Chelly, Teddy Furon, Stéphane Girard, Michael E Houle, Ken-ichi Kawarabayashi, and Michael Nett · 2018
Earlier work this paper cites.
Deep clustering for unsupervised learning of visual features
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Cited alongside, same era.
Intrinsic dimension of data representations in deep neural networks
Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan · 2019
Cited alongside, same era.
On the intrinsic dimensionality of image representations
Sixue Gong, Vishnu Naresh Boddeti, and Anil K Jain · 2019
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova · 2019
Cited alongside, same era.
Decoupled weight decay regularization
Barlow twins: Self-supervised learning via redundancy reduction
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny · 2021
Later among the works it cites.
Zero-cl: Instance and feature decorrelation for negative-free symmetric contrastive learning
Shaofeng Zhang, Feng Zhu, Junchi Yan, Rui Zhao, and Xiaokang Yang · 2021
Later among the works it cites.
Local intrinsic dimensionality, entropy and statistical divergences
James Bailey, Michael E Houle, and Xingjun Ma · 2022
Later among the works it cites.
BEit: BERT pre-training of image transformers
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei · 2022
Later among the works it cites.
VICReg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2022
Later among the works it cites.
Toward a geometrical understanding of self-supervised contrastive learning
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ilya Loshchilov and Frank Hutter · 2019
Cited alongside, same era.
Clustering financial return distributions using the fisher information metric
Stephen Taylor · 2019
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin · 2020
Cited alongside, same era.
Bootstrap your own latent-a new approach to self-supervised learning
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al · 2020
Cited alongside, same era.
Momentum contrast for unsupervised visual representation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick · 2020
Cited alongside, same era.
Emerging properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin · 2021
Cited alongside, same era.
Exploring simple siamese representation learning
Xinlei Chen and Kaiming He · 2021
Cited alongside, same era.
Romain Cosentino, Anirvan Sengupta, Salman Avestimehr, Mahdi Soltanolkotabi, Antonio Ortega, Ted Willke, and Mariano Tepper · 2022
Later among the works it cites.
Collider: A robust training framework for backdoor data
Hadi Mohaghegh Dolatabadi, Sarah Erfani, and Christopher Leckie · 2022
Later among the works it cites.
Improving self-supervised learning by characterizing idealized representations
Yann Dubois, Stefano Ermon, Tatsunori B Hashimoto, and Percy S Liang · 2022
Later among the works it cites.
Exploring the gap between collapsed & whitened features in self-supervised learning
Bobby He and Mete Ozay · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick · 2022
Later among the works it cites.
Understanding dimensional collapse in contrastive self-supervised learning
Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian · 2022
Later among the works it cites.
Understanding collapse in non-contrastive siamese representation learning
Alexander C Li, Alexei A Efros, and Deepak Pathak · 2022
Later among the works it cites.
Unsupervised visual representation learning by synchronous momentum grouping
Bo Pang, Yifan Zhang, Yaoyi Li, Jia Cai, and Cewu Lu · 2022
Later among the works it cites.
On the importance of hyperparameters and data augmentation for self-supervised learning
Diane Wagner, Fabio Ferreira, Danny Stoll, Robin Tibor Schirrmeister, Samuel Müller, and Frank Hutter · 2022
Later among the works it cites.
Simmim: A simple framework for masked image modeling
Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu · 2022
Later among the works it cites.
Decoupled contrastive learning
Chun-Hsiao Yeh, Cheng-Yao Hong, Yen-Chi Hsu, Tyng-Luh Liu, Yubei Chen, and Yann LeCun · 2022
Later among the works it cites.
How mask matters: Towards theoretical understandings of masked autoencoders
Qi Zhang, Yifei Wang, and Yisen Wang · 2022
Later among the works it cites.
Soft neighbors are positive supporters in contrastive visual representation learning
Chongjian Ge, Jiangliu Wang, Zhan Tong, Shoufa Chen, Yibing Song, and Ping Luo · 2023
Later among the works it cites.
Towards the generalization of contrastive self-supervised learning
Weiran Huang, Mingyang Yi, Xuyang Zhao, and Zihao Jiang · 2023
Later among the works it cites.
Mosaic representation learning for self-supervised visual pre-training
Zhaoqing Wang, Ziyu Chen, Yaqian Li, Yandong Guo, Jun Yu, Mingming Gong, and Tongliang Liu · 2023
Later among the works it cites.
Towards a unified theoretical understanding of non-contrastive learning via rank differential mechanism
Zhijian Zhuo, Yifei Wang, Jinwen Ma, and Yisen Wang · 2023
Later among the works it cites.