Fetching the paper…
Reading the bibliography…
We describe a minimalistic and interpretable method for unsupervised learning, without resorting to data augmentation, hyperparameter tuning, or other engineering designs, that achieves performance close to the SOTA SSL methods.
A theoretical analysis of contrastive unsupervised representation learning
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi · 1902
Earlier work this paper cites.
Algebraic connectivity of graphs
Miroslav Fiedler · 1973
Earlier work this paper cites.
Some demonstrations of the effects of structural descriptions in mental imagery
Geoffrey Hinton · 1979
Earlier work this paper cites.
Vector quantization
Robert M. Gray · 1984
Earlier work this paper cites.
Learning representations by back-propagating errors
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1986
Earlier work this paper cites.
Orthonormal bases of compactly supported wavelets
Ingrid Daubechies · 1988
Earlier work this paper cites.
A theory for multiresolution signal decomposition: the wavelet representation
Stephane G Mallat · 1989
Earlier work this paper cites.
Indexing by latent semantic analysis
Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman · 1990
Earlier work this paper cites.
Mapping part-whole hierarchies into connectionist networks
Geoffrey E Hinton · 1990
Earlier work this paper cites.
The design and use of steerable filters
William T Freeman, Edward H Adelson, et al · 1991
Earlier work this paper cites.
Shiftable multiscale transforms
Eero P Simoncelli, William T Freeman, Edward H Adelson, and David J Heeger · 1992
Earlier work this paper cites.
Basis pursuit
Shaobing Chen and David Donoho · 1994
Earlier work this paper cites.
Learning classification with unlabeled data
Virginia R de Sa · 1994
Earlier work this paper cites.
An information-maximization approach to blind separation and blind deconvolution
Anthony J Bell and Terrence J Sejnowski · 1995
Earlier work this paper cites.
The steerable pyramid: A flexible architecture for multi-scale derivative computation
Eero P Simoncelli and William T Freeman · 1995
Earlier work this paper cites.
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Bruno A Olshausen and David J Field · 1996
Earlier work this paper cites.
A fast fixed-point algorithm for independent component analysis
Aapo Hyvärinen and Erkki Oja · 1997
Earlier work this paper cites.
Sparse coding with an overcomplete basis set: A strategy employed by v1?
Bruno A Olshausen and David J Field · 1997
Earlier work this paper cites.
Nonlinear dimensionality reduction by locally linear embedding
Sam T Roweis and Lawrence K Saul · 2000
Earlier work this paper cites.
Normalized cuts and image segmentation
Jianbo Shi and Jitendra Malik · 2000
Earlier work this paper cites.
A global geometric framework for nonlinear dimensionality reduction
Joshua B Tenenbaum, Vin De Silva, and John C Langford · 2000
Earlier work this paper cites.
Atomic decomposition by basis pursuit
Scott Shaobing Chen, David L Donoho, and Michael A Saunders · 2001
Earlier work this paper cites.
Concept decompositions for large sparse text data using clustering
Inderjit S. Dhillon and Dharmendra S. Modha · 2001
Earlier work this paper cites.
Topographic independent component analysis
Aapo Hyvärinen, Patrik O Hoyer, and Mika Inki · 2001
Earlier work this paper cites.
A random walks view of spectral segmentation
Marina Meilă and Jianbo Shi · 2001
Earlier work this paper cites.
On spectral clustering: Analysis and an algorithm
Andrew Ng, Michael Jordan, and Yair Weiss · 2001
Earlier work this paper cites.
Natural image statistics and neural representation
Eero P Simoncelli and Bruno A Olshausen · 2001
Earlier work this paper cites.
Stochastic neighbor embedding
Geoffrey E Hinton and Sam Roweis · 2002
Earlier work this paper cites.
Natural image statistics and divisive normalization: Modeling nonlinearity and adaptation in cortical neurons
Martin J. Wainwright, Odelia Schwartz, and Eero P. Simoncelli · 2002
Earlier work this paper cites.
Slow feature analysis: Unsupervised learning of invariances
Laurenz Wiskott and Terrence J Sejnowski · 2002
Earlier work this paper cites.
Laplacian eigenmaps for dimensionality reduction and data representation
Mikhail Belkin and Partha Niyogi · 2003
Earlier work this paper cites.
Improved baselines with momentum contrastive learning
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He · 2003
Earlier work this paper cites.
Lower bounds for the partitioning of graphs
William E Donath and Alan J Hoffman · 2003
Earlier work this paper cites.
Bubbles: a unifying framework for low-level statistical properties of natural image sequences
Aapo Hyvärinen, Jarmo Hurri, and Jaakko Väyrynen · 2003
Earlier work this paper cites.
The nonlinear statistics of high-contrast patches in natural images
Ann B Lee, Kim S Pedersen, and David Mumford · 2003
Earlier work this paper cites.
Think globally, fit locally: unsupervised learning of low dimensional manifolds
Lawrence K Saul and Sam T Roweis · 2003
Earlier work this paper cites.
Multiclass spectral clustering
X Yu Stella and Jianbo Shi · 2003
Earlier work this paper cites.
Topological estimation using witness complexes
Vin De Silva and Gunnar E Carlsson · 2004
Earlier work this paper cites.
Sparse multidimensional scaling using landmark points
Vin De Silva and Joshua B Tenenbaum · 2004
Cited alongside, same era.
Latent semantic analysis
Susan T Dumais · 2004
Cited alongside, same era.
A tutorial on support vector regression
Alex J Smola and Bernhard Schölkopf · 2004
Cited alongside, same era.
How close are we to understanding v1?
Bruno A Olshausen and David J Field · 2005
Cited alongside, same era.
Pattern theory: from representation to inference
Ulf Grenander and Michael I Miller · 2006
Cited alongside, same era.
Dimensionality reduction by learning an invariant mapping
Raia Hadsell, Sumit Chopra, and Yann LeCun · 2006
Cited alongside, same era.
Pointer sentinel mixture models
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher · 2017
Later among the works it cites.
The sparse manifold transform
Yubei Chen, Dylan M. Paiton, and Bruno A. Olshausen · 2018
Later among the works it cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Later among the works it cites.
Representation learning with contrastive predictive coding
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Later among the works it cites.
Unsupervised feature learning via non-parametric instance discrimination
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce · 2006
Cited alongside, same era.
Selecting landmark points for sparse manifold learning
Jorge Silva, Jorge Marques, and João Lemos · 2006
Cited alongside, same era.
Untangling invariant object recognition
James J DiCarlo and David D Cox · 2007
Cited alongside, same era.
Semantic hierarchies for recognizing objects and parts
Boris Epshtein and Shimon Ullman · 2007
Cited alongside, same era.
Object recognition and segmentation by a fragment-based hierarchy
Shimon Ullman · 2007
Cited alongside, same era.
On the local behavior of spaces of natural images
Gunnar Carlsson, Tigran Ishkhanov, Vin De Silva, and Afra Zomorodian · 2008
Cited alongside, same era.
Wieland Brendel and Matthias Bethge · 2019
Later among the works it cites.
Kawin Ethayarajh · 2019
Later among the works it cites.
Perceptual straightening of natural videos
Olivier J Hénaff, Robbe LT Goris, and Eero P Simoncelli · 2019
Later among the works it cites.
Enhanced convolutional neural tangent kernels
Zhiyuan Li, Ruosong Wang, Dingli Yu, Simon S Du, Wei Hu, Ruslan Salakhutdinov, and Sanjeev Arora · 2019
Later among the works it cites.
Do imagenet classifiers generalize to imagenet?
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar · 2019
Later among the works it cites.
Word embedding visualization via dictionary learning
Juexiao Zhang, Yubei Chen, Brian Cheung, and Bruno A Olshausen · 2019
Later among the works it cites.
Momentum contrast for unsupervised visual representation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick · 2020
Later among the works it cites.
Neural kernels without tangents
Vaishaal Shankar, Alex Fang, Wenshuo Guo, Sara Fridovich-Keil, Jonathan Ragan-Kelley, Ludwig Schmidt, and Benjamin Recht · 2020
Later among the works it cites.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2021
Later among the works it cites.
High fidelity visualization of what your self-supervised representation knows about
Florian Bordes, Randall Balestriero, and Pascal Vincent · 2021
Later among the works it cites.
Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers
Hila Chefer, Shir Gur, and Lior Wolf · 2021
Later among the works it cites.
Exploring simple siamese representation learning
Xinlei Chen and Kaiming He · 2021
Later among the works it cites.
Provable guarantees for self-supervised deep learning with spectral contrastive loss
Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma · 2021
Later among the works it cites.
Primary visual cortex straightens natural video trajectories
Olivier J Hénaff, Yoon Bai, Julie A Charlton, Ian Nauhaus, Eero P Simoncelli, and Robbe LT Goris · 2021
Later among the works it cites.
How to represent part-whole hierarchies in a neural network
Geoffrey Hinton · 2021
Later among the works it cites.
Detecting formal thought disorder by deep contextualized word representations
Justyna Sarzynska-Wawer, Aleksander Wawer, Aleksandra Pawlak, Julia Szymanowska, Izabela Stefaniak, Michal Jarkiewicz, and Lukasz Okruszek · 2021
Later among the works it cites.
The unreasonable effectiveness of patches in deep convolutional kernels methods
Louis Thiry, Michael Arbel, Eugene Belilovsky, and Edouard Oyallon · 2021
Later among the works it cites.
Understanding self-supervised learning dynamics without contrastive pairs
Yuandong Tian, Xinlei Chen, and Surya Ganguli · 2021
Later among the works it cites.
Decoupled contrastive learning
Chun-Hsiao Yeh, Cheng-Yao Hong, Yen-Chi Hsu, Tyng-Luh Liu, Yubei Chen, and Yann LeCun · 2021
Later among the works it cites.
Zeyu Yun, Yubei Chen, Bruno A Olshausen, and Yann LeCun · 2021
Later among the works it cites.
Barlow twins: Self-supervised learning via redundancy reduction
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny · 2021
Later among the works it cites.
Randall Balestriero and Yann LeCun · 2022
Closest in time.
Redunet: A white-box deep network from the principle of maximizing rate reduction
Kwan Ho Ryan Chan, YD Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma · 2022
Closest in time.
Intra-instance vicreg: Bag of self-supervised image patch embedding
Yubei Chen, Adrien Bardes, Zengyi Li, and Yann LeCun · 2022
Closest in time.
On the duality between contrastive and non-contrastive self-supervised learning
Quentin Garrido, Yubei Chen, Adrien Bardes, Laurent Najman, and Yann Lecun · 2022
Closest in time.
Revisiting the critical factors of augmentation-invariant representation learning
Junqiang Huang, Xiangwen Kong, and Xiangyu Zhang · 2022
Closest in time.
Li Jing, Jiachen Zhu, and Yann LeCun · 2022
Closest in time.
A path towards autonomous machine intelligence
Y LeCun · 2022
Closest in time.
Neural manifold clustering and embedding
Zengyi Li, Yubei Chen, Yann LeCun, and Friedrich T Sommer · 2022
Closest in time.
On the principles of parsimony and self-consistency for the emergence of intelligence
Yi Ma, Doris Tsao, and Heung-Yeung Shum · 2022
Closest in time.
Human Language Understanding & Reasoning
Christopher D. Manning · 2022
Closest in time.
What do we maximize in self-supervised learning?
Ravid Shwartz-Ziv, Randall Balestriero, and Yann LeCun · 2022
Closest in time.