Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters

A mathematical theory of communication

C. E. Shannon · 1948

Earlier work this paper cites.

A method for the construction of minimum-redundancy codes

David A Huffman · 1952

Earlier work this paper cites.

Optimal brain damage

Y. LeCun, J. S. Denker, and S. A. Solla · 1990

Earlier work this paper cites.

Keeping the neural networks simple by minimizing the description length of the weights

G. E. Hinton and D. Van Camp · 1993

Earlier work this paper cites.

Autoencoders, minimum description length and helmholtz free energy

G. E. Hinton and R. S. Zemel · 1994

Earlier work this paper cites.

Efficient stochastic source coding and an application to a bayesian network source model

B. J. Frey and G. E. Hinton · 1997

Earlier work this paper cites.

The minimum description length principle

P. D. Grünwald · 2007

Cited alongside, same era.

The communication complexity of correlation

P. Harsha, R. Jain, D. McAllester, and J. Radhakrishnan · 2010

Cited alongside, same era.

An introduction to Kolmogorov complexity and its applications , volume 34

P. M. B. Vitanyi and M. Li · 2010

Cited alongside, same era.

Adam: A method for stochastic optimization

Original

D. P. Kingma and J. Ba · 2014

Cited alongside, same era.

Compressing neural networks with the hashing trick

W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen · 2015

Cited alongside, same era.

Learning both weights and connections for efficient neural network

Song Han, Jeff Pool, John Tran, and William Dally · 2015

Cited alongside, same era.

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

Song Han, Huizi Mao, and William J Dally · 2016

Later among the works it cites.

Beta-VAE: Learning basic visual concepts with a constrained variational framework

I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner · 2017

Later among the works it cites.

Bayesian compression for deep learning

C. Louizos, K. Ullrich, and M. Welling · 2017

Later among the works it cites.

The sample size required in importance sampling

S. Chatterjee and P. Diaconis · 2018

Closest in time.

Weightless: Lossy weight encoding for deep neural network compression

B. Reagen, U. Gupta, R. Adolf, M. M. Mitzenmacher, A. M. Rush, G.-Y. Wei, and D. Brooks · 2018

Closest in time.