Fetching the paper…
Reading the bibliography…
Contrastive learning (CL) has emerged as a powerful technique for representation learning, with or without label supervision.
A theoretical analysis of contrastive unsupervised representation learning
Arora, S., Khandeparkar, H., Khodak, M., Plevrakis, O., and Saunshi, N · 1902
Earlier work this paper cites.
Sparse coding with an overcomplete basis set: A strategy employed by v1?
Olshausen, B. A. and Field, D. J · 1997
Earlier work this paper cites.
Adaptive estimation of a quadratic functional by model selection
Laurent, B. and Massart, P · 2000
Earlier work this paper cites.
Probability for statisticians , volume 951
Shorack, G. R. and Shorack, G · 2000
Earlier work this paper cites.
Sparse coding and decorrelation in primary visual cortex during natural vision
Vinje, W. E. and Gallant, J. L · 2000
Earlier work this paper cites.
A Simple Framework for Contrastive Learning of Visual Representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G · 2002
Earlier work this paper cites.
Sparse coding in the primate cortex
Foldiak, P · 2003
Earlier work this paper cites.
Bootstrap your own latent: A new approach to self-supervised Learning, September 2020
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., Piot, B., Kavukcuoglu, K., Munos, R., and Valko, M · 2006
Earlier work this paper cites.
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., and Teh, Y.-W · 2006
Earlier work this paper cites.
Efficient learning of sparse representations with an energy-based model
Ranzato, M., Poultney, C., Chopra, S., and Cun, Y · 2006
Earlier work this paper cites.
Image sequence denoising via sparse and redundant representations
Protter, M. and Elad, M · 2008
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Linear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., and Huang, T · 2009
Earlier work this paper cites.
Addressing feature suppression in unsupervised visual representations, 2020
Li, T., Fan, L., Yuan, Y., He, H., Tian, Y., Feris, R., Indyk, P., and Katabi, D · 2012
Earlier work this paper cites.
Sparse modeling for image and vision processing
Mairal, J., Bach, F., Ponce, J., et al · 2014
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Implicit regularization in matrix factorization
Gunasekar, S., Woodworth, B. E., Bhojanapalli, S., Neyshabur, B., and Srebro, N · 2017
Earlier work this paper cites.
Convolutional neural networks analyzed via convolutional sparse coding
Papyan, V., Romano, Y., and Elad, M · 2017
Cited alongside, same era.
Linear algebraic structure of word senses, with applications to polysemy
Arora, S., Li, Y., Liang, Y., Ma, T., and Risteski, A · 2018
Cited alongside, same era.
Implicit bias of gradient descent on linear convolutional networks
Gunasekar, S., Lee, J. D., Soudry, D., and Srebro, N · 2018
Cited alongside, same era.
The implicit bias of gradient descent on separable data
Soudry, D., Hoffer, E., Nacson, M. S., Gunasekar, S., and Srebro, N · 2018
Cited alongside, same era.
The implicit bias of gradient descent on nonseparable data
Ji, Z. and Telgarsky, M · 2019
Cited alongside, same era.
Sgd on neural networks learns functions of increasing complexity
Kalimeris, D., Kaplun, G., Nakkiran, P., Edelman, B., Yang, T., Barak, B., and Zhang, H · 2019
Dissecting supervised constrastive learning
Graf, F., Hofer, C., Niethammer, M., and Kwitt, R · 2021
Later among the works it cites.
Neural collapse under mse loss: Proximity to and dynamics on the central path
Han, X., Papyan, V., and Donoho, D. L · 2021
Later among the works it cites.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T · 2021
Later among the works it cites.
A broad study on the transferability of visual representations with contrastive learning
Islam, A., Chen, C.-F. R., Panda, R., Karlinsky, L., Radke, R., and Feris, R · 2021
Later among the works it cites.
The power of contrast for feature learning: A theoretical analysis
Ji, W., Deng, Z., Nakada, R., Zou, J., and Zhang, L · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Towards understanding the generalization bias of two layer convolutional linear classifiers with gradient descent
Wu, Y., Poczos, B., and Singh, A · 2019
Cited alongside, same era.
Debiased Contrastive Learning
Chuang, C.-Y., Robinson, J., Lin, Y.-C., Torralba, A., and Jegelka, S · 2020
Cited alongside, same era.
Supervised contrastive learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D · 2020
Cited alongside, same era.
Prevalence of neural collapse during the terminal phase of deep learning training
Papyan, V., Han, X., and Donoho, D. L · 2020
Cited alongside, same era.
Implicit regularization in deep learning may not be explainable by norms
Razin, N. and Cohen, N · 2020
Cited alongside, same era.
An investigation of why overparameterization exacerbates spurious correlations
Sagawa, S., Raghunathan, A., Koh, P. W., and Liang, P · 2020
Cited alongside, same era.
Predicting what you already know helps: Provable self-supervised learning
Lee, J. D., Lei, Q., Saunshi, N., and Zhuo, J · 2021
Later among the works it cites.
Self-supervised learning is more robust to dataset imbalance
Liu, H., HaoChen, J. Z., Gaidon, A., and Ma, T · 2021
Later among the works it cites.
Gradient descent on two-layer nets: Margin maximization and simplicity bias
Lyu, K., Li, Z., Wang, R., and Arora, S · 2021
Later among the works it cites.
Can contrastive learning avoid shortcut solutions?, 2021
Robinson, J., Sun, L., Yu, K., Batmanghelich, K., Jegelka, S., and Sra, S · 2021
Later among the works it cites.
Toward understanding the feature learning process of self-supervised contrastive learning
Wen, Z. and Li, Y · 2021
Later among the works it cites.
A geometric analysis of neural collapse with unconstrained features
Zhu, Z., Ding, T., Zhou, J., Li, X., You, C., Sulam, J., and Qu, Q · 2021
Later among the works it cites.
Understanding the generalization of adam in learning neural networks with proper regularization
Zou, D., Cao, Y., Li, Y., and Gu, Q · 2021
Later among the works it cites.
Perfectly balanced: Improving transfer and robustness of supervised contrastive learning
Chen, M., Fu, D. Y., Narayan, A., Zhang, M., Song, Z., Fatahalian, K., and Ré, C · 2022
Later among the works it cites.
A theoretical study of inductive biases in contrastive learning
HaoChen, J. Z. and Ma, T · 2022
Later among the works it cites.
Limitations of neural collapse for understanding generalization in deep learning
Hui, L., Belkin, M., and Nakkiran, P · 2022
Later among the works it cites.
Neural collapse under cross-entropy loss
Lu, J. and Steinerberger, S · 2022
Later among the works it cites.
Understanding contrastive learning requires incorporating inductive biases
Saunshi, N., Ash, J., Goel, S., Misra, D., Zhang, C., Arora, S., Kakade, S., and Krishnamurthy, A · 2022
Later among the works it cites.