Fetching the paper…

Reading the bibliography…

2019

On the Estimation of Mutual Information

Carrara, Nicholas, Ernst, Jesse

Original
PDF
Post on X

Understand

In this paper we focus on the estimation of mutual information from finite samples $(\mathcal{X}\times\mathcal{Y})$.

The main concern with estimations of mutual information is their robustness under the class of transformations for which it remains invariant: i.e.
type I (coordinate transformations), III (marginalizations) and special cases of type IV (embeddings, products).
Estimators which fail to meet these standards are not \textit{robust} in their general applicability.

On the Estimation of Mutual Information · Around

Built on

The design of mutual information and it’s applications
Original
N. Carrara · 1910
Earlier work this paper cites.
The information bottleneck method
Naftali Tishby, Fernando C. Pereira, and William Bialek · 2000
Earlier work this paper cites.
Estimating mutual information
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger · 2004
Earlier work this paper cites.

Similar

Elements of information theory
T. M. Cover and Joy A. Thomas · 2006
Cited alongside, same era.
Efficient Estimation of Mutual Information for Strongly Dependent Variables
Original
Shuyang Gao, Greg Ver Steeg, and Aram Galstyan · 2014
Cited alongside, same era.
Deep Learning and the Information Bottleneck Principle
Original
Naftali Tishby and Noga Zaslavsky · 2015
Cited alongside, same era.

Then

The Information Sieve
Original
Greg Ver Steeg and Aram Galstyan · 2015
Later among the works it cites.
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
Original
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel · 2016
Later among the works it cites.
On the Upper Limit of Separability
Original
Nicholas Carrara and Jesse A. Ernst · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…