Entropy-Constrained Training of Deep Neural Networks

J. Rissanen, “Paper: Modeling by shortest data description,” Automatica

1978

Earlier work this paper cites.

Y. L. Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in Neural Information Processing Systems (NIPS)

1990

Earlier work this paper cites.

B. Hassibi, D. G. Stork, and G. J. Wolff, “Optimal brain surgeon and general network pruning,” in IEEE International Conference on Neural Networks

1993

Earlier work this paper cites.

G. E. Hinton and D. van Camp, “Keeping the neural networks simple by minimizing the description length of the weights,” in Conference on Computational Learning Theory (COLT)

1993

Earlier work this paper cites.

Q. Xie and A. R. Barron, “Asymptotic minimax regret for data compression, gambling, and prediction,” IEEE Transactions on Information Theory

2000

Earlier work this paper cites.

C. E. Shannon, “A mathematical theory of communication,” SIGMOBILE Mobile Computing and Communications Review

2001

Earlier work this paper cites.

H. V. Antti Honkela, “Variational learning and bits-back coding: An information-theoretic view to bayesian learning,” 2004

2004

Earlier work this paper cites.

New York, NY, USA: Wiley-Interscience, 2006

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) · 2006

Earlier work this paper cites.

A. Barron, J. Rissanen, and B. Yu, “The minimum description length principle in coding and modeling,” IEEE Transactions on Information Theory

2006

Earlier work this paper cites.

Adaptive computation and machine learning, MIT Press, 2007

P. Grünwald and J. Rissanen, The Minimum Description Length Principle · 2007

Earlier work this paper cites.

T. Wiegand and H. Schwarz, “Source coding: Part 1 of fundamentals of source and video coding,” Found. Trends Signal Process

2011

Earlier work this paper cites.

Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Müller, “Efficient backprop,” in Neural Networks: Tricks of the Trade - Second Edition, Springer LNCS 7700

2012

Earlier work this paper cites.

Y. LeCun, Y. Bengio, and G. E. Hinton, “Deep learning,” Nature

2015

Earlier work this paper cites.

S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” arXiv:1510.00149

Original

2015

Cited alongside, same era.

D. P. Kingma, T. Salimans, and M. Welling, “Variational dropout and the local reparameterization trick,” in Advances in Neural Information Processing Systems (NIPS)

2015

Cited alongside, same era.

S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural networks,” in Advances in Neural Information Processing Systems (NIPS)

2015

Cited alongside, same era.

M. Courbariaux, Y. Bengio, and J. David, “Binaryconnect: Training deep neural networks with binary weights during propagations,” arXiv:1511.00363

Original

2015

Cited alongside, same era.

A. Achille and S. Soatto, “On the emergence of invariance and disentangling in deep representations,” arXiv:1706.01350

Original

2017

Later among the works it cites.

D. Molchanov, A. Ashukha, and D. Vetrov, “Variational dropout sparsifies deep neural networks,” in International Conference on Machine Learning (ICML)

2017

Later among the works it cites.

C. Louizos, K. Ullrich, and M. Welling, “Bayesian Compression for Deep Learning,” in Advances in Neural Information Processing Systems (NIPS)

2017

Later among the works it cites.

C. Louizos, M. Welling, and D. P. Kingma, “Learning Sparse Neural Networks through L ​ _ ​ 0 L\_0 Regularization,” arXiv:1712.01312

Original

2017

Later among the works it cites.

O. Shayer, D. Levi, and E. Fetaya, “Learning discrete weights using the local reparameterization trick,” arXiv:1710.07739