Nonlinear Information Bottleneck

Matthew Chalk, Olivier Marre, and Gasper Tkacik, “Relevant sparse codes with variational information bottleneck,” in Advances in Neural Information Processing Systems (2016) pp. 1957–1965

1965

Earlier work this paper cites.

H. Witsenhausen and A. Wyner, “A conditional entropy bound for a pair of discrete random variables,” IEEE Transactions on Information Theory 21

1975

Earlier work this paper cites.

Rudolf Ahlswede and János Körner, “Source coding with Side Information and a Converse for Degraded Broadcast Channels,” IEEE Transaction on Information Theory , 9 (1975)

1975

Earlier work this paper cites.

Geoffrey E. Hinton and Richard S. Zemel, “Autoencoders, minimum description length, and Helmholtz free energy,” Advances in neural information processing systems , 3–3 (1994)

1994

Earlier work this paper cites.

Nicol Norbert Schraudolph, Optimization of entropy with neural networks , Ph.D. thesis, Citeseer (1995)

1995

Earlier work this paper cites.

Geoffrey E. Hinton and Richard S. Zemel, “Minimizing description length in an unsupervised neural network,” Preprint (1997)

1997

Earlier work this paper cites.

R Kelley Pace and Ronald Barry, “Sparse spatial autoregressions,” Statistics & Probability Letters 33

1997

Earlier work this paper cites.

Kaisa Miettinen, Nonlinear Multiobjective Optimization , edited by Frederick S. Hillier, International Series in Operations Research & Management Science, Vol. 12 (Springer US, Boston, MA, 1998)

1998

Earlier work this paper cites.

N. Tishby, F. Pereira, and W. Bialek, “The information bottleneck method,” in 37th Allerton Conf on Communication (1999)

1999

Earlier work this paper cites.

Noam Slonim and Naftali Tishby, “Document clustering using word clusters via the information bottleneck method,” in Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval (ACM, 2000) pp. 208–215

2000

Earlier work this paper cites.

Alexander G. Dimitrov and John P. Miller, “Neural coding and decoding: communication channels and quantization,” Network: Computation in Neural Systems 12

2001

Earlier work this paper cites.

Naftali Tishby and Noam Slonim, “Data clustering by markovian relaxation and the information bottleneck method,” in Advances in neural information processing systems (2001) pp. 640–646

2001

Earlier work this paper cites.

Claude Lemaréchal, “Lagrangian relaxation,” in Computational combinatorial optimization (Springer, 2001) pp. 112–156

2001

Earlier work this paper cites.

Inés Samengo, “Information loss in an optimal maximum likelihood decoding,” Neural computation 14

2002

Earlier work this paper cites.

Ran Gilad-Bachrach, Amir Navot, and Naftali Tishby, “An Information Theoretic Tradeoff between Complexity and Accuracy,” in Learning Theory and Kernel Machines , Vol. 2777, edited by Gerhard Goos, Juris Hartmanis, Jan van Leeuwen, Bernhard Schölkopf, and Manfred K. Warmuth (Springer Berlin Heidelberg, 2003) pp. 595–609

2003

Earlier work this paper cites.

Jean Cardinal, “Compression of side information,” in icme (IEEE, 2003) pp. 569–572

2003

Earlier work this paper cites.

Kari Torkkola, “Feature extraction by non-parametric mutual information maximization,” Journal of machine learning research 3

2003

Earlier work this paper cites.

Nicol N. Schraudolph, “Gradient-based manipulation of nonparametric entropy estimates,” Neural Networks, IEEE Transactions on 15

2004

Earlier work this paper cites.

J Winn, A Criminisi, and T Minka, “Object categorization by learned universal visual dictionary,” in Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1 , Vol. 2 (IEEE, 2005) pp. 1800–1807

2005

Earlier work this paper cites.

Gal Chechik, Amir Globerson, Naftali Tishby, and Yair Weiss, “Information bottleneck for Gaussian variables,” Journal of Machine Learning Research 6

2005

Cited alongside, same era.

Sarit Shwartz, Michael Zibulevsky, and Yoav Y. Schechner, “Fast kernel entropy estimation and optimization,” Signal Processing 85

2005

Cited alongside, same era.

Katerina Hlavávcková-Schindler, Milan Palus, Martin Vejmelka, and Joydeep Bhattacharya, “Causality detection based on information-theoretic approaches in time series analysis,” Physics Reports 441

2007

Cited alongside, same era.

Georg Zeitler, Ralf Koetter, Gerhard Bauch, and Joerg Widmer, “Design of network coding functions in multihop relay networks,” in Turbo Codes and Related Topics, 2008 5th International Symposium on (Citeseer, 2008) pp. 249–254

2008

Cited alongside, same era.

Ravid Shwartz-Ziv and Naftali Tishby, “Opening the black box of deep neural networks via information,” arXiv preprint arXiv:1703.00810 (2017)

Original

2017

Closest in time.

Artemy Kolchinsky and Brendan D. Tracey, “Estimating mixture entropy with pairwise distances,” Entropy 19

2017

Closest in time.

Artemy Kolchinsky, Brendan D. Tracey, and David H. Wolpert, “Nonlinear information bottleneck (v1),” arXiv e-prints abs/1705.02436v1

Original

2017

Closest in time.

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework.” ICLR 2

2017

Closest in time.

Han Xiao, Kashif Rasul, and Roland Vollgraf, “Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 (2017)