Fetching the paper…
Reading the bibliography…
In this paper, we examine self-supervised learning methods, particularly VICReg, to provide an information-theoretical understanding of their construction.
An omnibus test of normality for moderate and large size samples
D’Agostino, R. B · 1971
Earlier work this paper cites.
Self-organization in a perceptual network
Linsker, R · 1988
Earlier work this paper cites.
Signature verification using a” siamese” time delay neural network
Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., and Shah, R · 1993
Earlier work this paper cites.
Identification of piecewise affine models in noisy environment
Fantuzzi, C., Simani, S., Beghelli, S., and Rovatti, R · 2002
Earlier work this paper cites.
On entropy approximation for gaussian mixture random vectors
Huber, M., Bailey, T., Durrant-Whyte, H., and Hanebeck, U · 2008
Earlier work this paper cites.
A course in approximation theory , volume 101
Cheney, E. W. and Light, W. A · 2009
Earlier work this paper cites.
Control theoretic splines: optimal control, statistics, and path planning
Egerstedt, M. and Martin, C · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A · 2009
Earlier work this paper cites.
Data spectroscopy: Eigenspaces of convolution operators and clustering
Shi, T., Belkin, M., and Yu, B · 2009
Earlier work this paper cites.
On the number of linear regions of deep neural networks
Montufar, G. F., Pascanu, R., Cho, K., and Bengio, Y · 2014
Earlier work this paper cites.
Deep variational information bottleneck
Alemi, A. A., Fischer, I., Dillon, J. V., and Murphy, K · 2016
Earlier work this paper cites.
Deep Learning , volume 1
Goodfellow, I., Bengio, Y., and Courville, A · 2016
Earlier work this paper cites.
Arbitrarily tight bounds on differential entropy of gaussian mixtures
Moshksar, K. and Khandani, A. K · 2016
Earlier work this paper cites.
Estimating mixture entropy with pairwise distances
Kolchinsky, A. and Tracey, B. D · 2017
Cited alongside, same era.
Opening the black box of deep neural networks via information
Shwartz-Ziv, R. and Tishby, N · 2017
Cited alongside, same era.
Information-theoretic analysis of generalization capability of learning algorithms
Xu, A. and Raginsky, M · 2017
Cited alongside, same era.
Information dropout: Learning optimal representations through noisy computation
Achille, A. and Soatto, S · 2018
Cited alongside, same era.
A spline theory of deep networks
Balestriero, R. and Baraniuk, R · 2018
Cited alongside, same era.
Unsupervised learning of visual features by contrasting cluster assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A · 2020
Later among the works it cites.
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G · 2020
Later among the works it cites.
Learning robust representations via multi-view information bottleneck
Federici, M., Dutta, A., Forré, P., Kushman, N., and Akata, Z · 2020
Later among the works it cites.
Bootstrap your own latent-a new approach to self-supervised learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al · 2020
Later among the works it cites.
Self-supervised learning of pretext-invariant representations
Misra, I. and Maaten, L. v. d · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Belghazi, M. I., Baratin, A., Rajeswar, S., Ozair, S., Bengio, Y., Courville, A., and Hjelm, R. D · 2018
Cited alongside, same era.
Estimating Information Flow in Neural Networks
Goldfeld, Z., van den Berg, E., Greenewald, K., Melnyk, I., Nguyen, N., Kingsbury, B., and Polyanskiy, Y · 2018
Cited alongside, same era.
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O · 2018
Cited alongside, same era.
Representation compression and generalization in deep neural networks, 2018
Shwartz-Ziv, R., Painsky, A., and Tishby, N · 2018
Cited alongside, same era.
Learning representations for neural network-based classification using the information bottleneck principle
Amjad, R. A. and Geiger, B. C · 2019
Cited alongside, same era.
A theoretical analysis of contrastive unsupervised representation learning
Arora, S., Khandeparkar, H., Khodak, M., Plevrakis, O., and Saunshi, N · 2019
Cited alongside, same era.
Piran, Z., Shwartz-Ziv, R., and Tishby, N · 2020
Later among the works it cites.
Information in infinite ensembles of infinitely-wide neural networks
Shwartz-Ziv, R. and Alemi, A. A · 2020
Later among the works it cites.
Reasoning about generalization via conditional mutual information
Steinke, T. and Zakynthinou, L · 2020
Later among the works it cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Bardes, A., Ponce, J., and LeCun, Y · 2021
Later among the works it cites.
Emerging properties in self-supervised vision transformers
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A · 2021
Later among the works it cites.
Exploring simple siamese representation learning
Chen, X. and He, K · 2021
Later among the works it cites.
Lossy compression for lossless prediction
Dubois, Y., Bloem-Reddy, B., Ullrich, K., and Maddison, C. J · 2021
Later among the works it cites.
Information flow in deep neural networks
Shwartz-Ziv, R · 2022
Closest in time.