Fetching the paper…
Reading the bibliography…
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data.
Self-organization in a perceptual network
Ralph Linsker · 1988
Earlier work this paper cites.
Self-organizing neural network that discovers surfaces in random-dot stereograms
Suzanna Becker and Geoffrey E Hinton · 1992
Earlier work this paper cites.
Unsupervised classifiers, mutual information and phantom targets
John S Bridle, Anthony JR Heading, and David JC MacKay · 1992
Earlier work this paper cites.
An information-maximization approach to blind separation and blind deconvolution
Anthony J Bell and Terrence J Sejnowski · 1995
Earlier work this paper cites.
The IM algorithm: a variational approach to information maximization
David Barber and Felix V Agakov · 2003
Earlier work this paper cites.
Estimating mutual information
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger · 2004
Earlier work this paper cites.
Discriminative clustering by regularized information maximization
Andreas Krause, Pietro Perona, and Ryan G Gomes · 2010
Earlier work this paper cites.
Estimating divergence functionals and the likelihood ratio by convex risk minimization
XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan · 2010
Earlier work this paper cites.
Scikit-learn: Machine learning in python
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al · 2011
Earlier work this paper cites.
Hamming distance metric learning
Mohammad Norouzi, David J Fleet, and Ruslan R Salakhutdinov · 2012
Earlier work this paper cites.
Tighter variational representations of f-divergences via restriction to probability measures
Avraham Ruderman, Mark D Reid, Darío García-García, and James Petterson · 2012
Earlier work this paper cites.
Deep canonical correlation analysis
Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu · 2013
Earlier work this paper cites.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien · 2014
Earlier work this paper cites.
Auto-encoding variational Bayes
Diederik P Kingma and Max Welling · 2014
Earlier work this paper cites.
Deep metric learning using triplet network
Elad Hoffer and Nir Ailon · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
Facenet: A unified embedding for face recognition and clustering
Florian Schroff, Dmitry Kalenichenko, and James Philbin · 2015
Cited alongside, same era.
Unsupervised learning of visual representations using videos
Xiaolong Wang and Abhinav Gupta · 2015
Cited alongside, same era.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton · 2016
Cited alongside, same era.
Density estimation using real nvp
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio · 2016
Cited alongside, same era.
f-GAN: Training generative neural samplers using variational divergence minimization
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka · 2016
Cited alongside, same era.
Representation learning with contrastive predictive coding
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Later among the works it cites.
A theoretical analysis of contrastive unsupervised representation learning
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi · 2019
Closest in time.
Learning representations by maximizing mutual information across views
Philip Bachman, R Devon Hjelm, and William Buchwalter · 2019
Closest in time.
Rethinking lossy compression: The rate-distortion-perception tradeoff
Yochai Blau and Tomer Michaeli · 2019
Closest in time.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Kihyuk Sohn · 2016
Cited alongside, same era.
Learning discrete representations via information maximizing self-augmented training
Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, and Masashi Sugiyama · 2017
Cited alongside, same era.
Cross-view asymmetric metric learning for unsupervised person re-identification
Hong-Xing Yu, Ancong Wu, and Wei-Shi Zheng · 2017
Cited alongside, same era.
Fixing a Broken ELBO
Alexander Alemi, Ben Poole, Ian Fischer, Joshua Dillon, Rif A Saurous, and Kevin Murphy · 2018
Cited alongside, same era.
Mutual information neural estimation
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Devon Hjelm, and Aaron Courville · 2018
Cited alongside, same era.
Zhuang Ma and Michael Collins · 2018
Cited alongside, same era.
Formal limitations on the measurement of mutual information
David McAllester and Karl Statos · 2018
Cited alongside, same era.
Olivier J Hénaff, Ali Razavi, Carl Doersch, SM Eslami, and Aaron van den Oord · 2019
Closest in time.
Learning deep representations by mutual information estimation and maximization
R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio · 2019
Closest in time.
Invariant information clustering for unsupervised image classification and segmentation
Xu Ji, João F Henriques, and Andrea Vedaldi · 2019
Closest in time.
Revisiting self-supervised visual representation learning
Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer · 2019
Closest in time.
Wasserstein dependency measure for representation learning
Sherjil Ozair, Corey Lynch, Yoshua Bengio, Aaron van den Oord, Sergey Levine, and Pierre Sermanet · 2019
Closest in time.
On variational bounds of mutual information
Ben Poole, Sherjil Ozair, Aaron van den Oord, Alex Alemi, and George Tucker · 2019
Closest in time.
Contrastive bidirectional transformer for temporal representation learning
Chen Sun, Fabien Baradel, Kevin Murphy, and Cordelia Schmid · 2019
Closest in time.
Yonglong Tian, Dilip Krishnan, and Phillip Isola · 2019
Closest in time.
Large-scale visual relationship understanding
Ji Zhang, Yannis Kalantidis, Marcus Rohrbach, Manohar Paluri, Ahmed Elgammal, and Mohamed Elhoseiny · 2019
Closest in time.
A Theory of Usable Information under Computational Constraints
Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon · 2020
Closest in time.