Fetching the paper…

Reading the bibliography…

2011

Learning Representations by Maximizing Compression

Gregor, Karol, LeCun, Yann

Original
PDF
Post on X

Understand

We give an algorithm that learns a representation of data through compression.

The algorithm 1) predicts bits sequentially from those previously seen and 2) has a structure and a number of computations similar to an autoencoder.
The likelihood under the model can be calculated exactly, and arithmetic coding can be used directly for compression.
When training on digits the algorithm learns filters similar to those of restricted boltzman machines and denoising autoencoders.

Learning Representations by Maximizing Compression · Around

Built on

Emergence of simple-cell receptive field properties by learning a sparse code for natural images
B.A. Olshausen et al · 1996
Earlier work this paper cites.
High quality document image compression with” djvu”
L. Bottou, P. Haffner, P.G. Howard, P. Simard, Y. Bengio, and Y. LeCun · 1998
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.

Similar

A fast learning algorithm for deep belief nets
G.E. Hinton, S. Osindero, and Y.W. Teh · 2006
Cited alongside, same era.
Estimation of non-normalized statistical models by score matching
A. Hyvarinen · 2006
Cited alongside, same era.
A novel connectionist system for unconstrained handwriting recognition
A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber · 2008
Cited alongside, same era.

Then

Extracting and composing robust features with denoising autoencoders
P. Vincent, H. Larochelle, Y. Bengio, and P.A. Manzagol · 2008
Later among the works it cites.
Herding dynamic weights for partially observed random field models
Original
M. Welling · 2009
Later among the works it cites.
Parametric herding
Y. Chen and M. Welling · 2010
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…