Fetching the paper…
Reading the bibliography…
Self-predictive unsupervised learning methods such as BYOL or SimSiam have shown impressive results, and counter-intuitively, do not collapse to trivial representations.
LIII. On lines and planes of closest fit to systems of points in space
Karl Pearson · 1901
Earlier work this paper cites.
Iterative berechung der reziproken matrix
Günther Schulz · 1933
Earlier work this paper cites.
A lower bound for the smallest eigenvalue of the Laplacian
Jeff Cheeger · 1969
Earlier work this paper cites.
The method of stochastic approximation for the determination of the least eigenvalue of a symmetrical matrix
T. P. Krasulina · 1969
Earlier work this paper cites.
The matrix sign function and computations in systems
Eugene D. Denman and Alex N. Beavers · 1976
Earlier work this paper cites.
Simplified neuron model as a principal component analyzer
Erkki Oja · 1982
Earlier work this paper cites.
Principal components, minor components, and linear neural networks
Erkki Oja · 1992
Earlier work this paper cites.
Signature Verification Using A ”Siamese” Time Delay Neural Network
J. Bromley, James W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, Eduard Säckinger, and R. Shah · 1993
Earlier work this paper cites.
Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values†
Pentti Paatero and Unto Tapper · 1994
Earlier work this paper cites.
Normalized cuts and image segmentation
Jianbo Shi and Jitendra Malik · 1997
Earlier work this paper cites.
The Geometry of Algorithms with Orthogonality Constraints
Alan Edelman, Tomás A. Arias, and S.T. Smith · 1998
Earlier work this paper cites.
Learning the parts of objects by non-negative matrix factorization
Daniel D. Lee and H. Sebastian Seung · 1999
Earlier work this paper cites.
Optimization Algorithms on Matrix Manifolds
Pierre-Antoine Absil, Robert E. Mahony, and Rodolphe Sepulchre · 2007
Earlier work this paper cites.
Sampling from large matrices: An approach through geometric functional analysis
M. Rudelson and R. Vershynin · 2007
Earlier work this paper cites.
Functions of Matrices - Theory and Computation
Nicholas John Higham · 2008
Earlier work this paper cites.
Trace optimization and eigenproblems in dimension reduction methods
Effrosini Kokiopoulou, Jie Chen, and Yousef Saad · 2011
Earlier work this paper cites.
Stochastic Gradient Descent on Riemannian Manifolds
Silvére Bonnabel · 2013
Earlier work this paper cites.
Matrix analysis
Roger A. Horn and Charles R. Johnson · 2013
Cited alongside, same era.
A stochastic PCA and SVD algorithm with an exponential convergence rate
Ohad Shamir · 2014
Cited alongside, same era.
On the matrix inversion approximation based on Neumann series in massive MIMO systems
Dengkui Zhu, Boyu Li, and Ping Liang · 2015
Cited alongside, same era.
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Streaming PCA: Matching matrix bernstein and near-optimal finite sample guarantees for oja’s algorithm
Prateek Jain, Chi Jin, Sham M. Kakade, Praneeth Netrapalli, and Aaron Sidford · 2016
Cited alongside, same era.
Large batch training of convolutional networks
Yang You, Igor Gitman, and Boris Ginsburg · 2017
Scaling up visual and vision-language representation learning with noisy text supervision
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig · 2021
Later among the works it cites.
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever · 2021
Later among the works it cites.
Understanding Self-Supervised Learning Dynamics without Contrastive Pairs
Yuandong Tian, Xinlei Chen, and S. Ganguli · 2021
Later among the works it cites.
Towards demystifying representation learning with non-contrastive self-supervision
Xiang Wang, Xinlei Chen, Simon Shaolei Du, and Yuandong Tian · 2021
Later among the works it cites.
Barlow Twins: Self-supervised learning via redundancy reduction
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
JAX: composable transformations of Python+NumPy programs, 2018
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, and Skye Wanderman-Milne · 2018
Cited alongside, same era.
SpectralNet: Spectral Clustering using Deep Neural Networks
Uri Shaham, Kelly Stanton, Haochao Li, Boaz Nadler, Ronen Basri, and Yuval Kluger · 2018
Cited alongside, same era.
Representation Learning with Contrastive Predictive Coding
Aäron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Cited alongside, same era.
Spectral Inference Networks: Unifying Deep and Spectral Learning
David Pfau, Stig Petersen, Ashish Agarwal, David G. T. Barrett, and Kimberly L. Stachenfeld · 2019
Cited alongside, same era.
Exponentially convergent stochastic k-PCA without variance reduction
Cheng Tang · 2019
Cited alongside, same era.
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton · 2020
Cited alongside, same era.
Later among the works it cites.
Fast and accurate optimization on the orthogonal manifold without retraction
Pierre Ablin and Gabriel Peyré · 2022
Later among the works it cites.
Flamingo: a Visual Language Model for Few-Shot Learning
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan · 2022
Later among the works it cites.
Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods
Randall Balestriero and Yann LeCun · 2022
Later among the works it cites.
VICReg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2022
Later among the works it cites.
Neural eigenfunctions are structured representation learners
Zhijie Deng, Jiaxin Shi, Hao Zhang, Peng Cui, Cewu Lu, and Jun Zhu · 2022
Later among the works it cites.
An eigenspace view reveals how predictor networks and stop-grads provide implicit variance regularization
Manu Srinath Halvagal, Axel Laborieux, and Friedemann Zenke · 2022
Later among the works it cites.
Bridging the gap from asymmetry tricks to decorrelation principles in non-contrastive self-supervised learning
Kang-Jun Liu, Masanori Suganuma, and Takayuki Okatani · 2022
Later among the works it cites.
Hierarchical text-conditional image generation with clip latents
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen · 2022
Later among the works it cites.
An investigation into whitening loss for self-supervised learning
Xi Weng, Lei Huang, Lei Zhao, Rao Muhammad Anwer, Salman Khan, and Fahad Shahbaz Khan · 2022
Later among the works it cites.
Zero-CL: Instance and feature decorrelation for negative-free symmetric contrastive learning
Shaofeng Zhang, Feng Zhu, Junchi Yan, Rui Zhao, and Xiaokang Yang · 2022
Later among the works it cites.
Towards a unified theoretical understanding of non-contrastive learning via rank differential mechanism
Anonymous · 2023
Closest in time.