Deep Learning and the Information Bottleneck Principle

K. Rose, E. Gurewitz, and G. C. Fox, “Statistical mechanics and phase transitions in clustering,” Phys. Rev. Lett. , vol. 65, pp. 945–948, 1990

Original

1990

Earlier work this paper cites.

T. Cover and J. Thomas, Elements of information theory . Wiley New York, 1991

1991

Earlier work this paper cites.

W. H. R. Equitz and T. M. Cover, “Successive refinement of information,” IEEE Transactions on Information Theory , vol. 37, no. 2, pp. 269–275, 1991

1991

Earlier work this paper cites.

Y. LeCun and Y. Bengio, “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks , vol. 3361, p. 310, 1995

1995

Earlier work this paper cites.

K. Rose, “Deterministic annealing for clustering, compression, classification, regression, and related optimization problems,” in Proceedings of the IEEE , 1998, pp. 2210–2239

1998

Earlier work this paper cites.

N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” in Proceedings of the 37-th Annual Allerton Conference on Communication, Control and Computing , 1999, pp. 368–377

1999

Cited alongside, same era.

R. Gilad-Bachrach, A. Navot, and N. Tishby, “An information theoretic tradeoff between complexity and accuracy,” in Proceedings of the COLT , 2003

2003

Cited alongside, same era.

G. Chechik, A. Globerson, N. Tishby, and Y. Weiss, “Information bottleneck for gaussian variables,” Journal of Machine Learning Research , vol. 6, pp. 165–188, 2005

2005

Cited alongside, same era.

G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science , vol. 313, no. 5786, pp. 504–507, July 2006

2006

Cited alongside, same era.

Y. Bengio, “Learning Deep Architectures for AI,” Foundations and Trends in Machine Learning , vol. 2, no. 1, pp. 1–127, 2009

O. Shamir, S. Sabato, and N. Tishby, “Learning and generalization with the information bottleneck,” Theor. Comput. Sci. , vol. 411, no. 29-30, pp. 2696–2711, 2010

2010

Later among the works it cites.

A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (NIPS) , 2012, pp. 1106–1114

2012

Later among the works it cites.

Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 35, no. 8, pp. 1798–828, Aug. 2013

2013

Later among the works it cites.

P. Mehta and D. J. Schwab, “An exact mapping between the variational renormalization group and deep learning,” CoRR , vol. abs/1410.3831, 2014

Original

2014

Later among the works it cites.