Applications of Common Entropy for Causal Inference

Understand

We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent.

• The minimum entropy required for such a latent is known as common entropy in information theory.
• We extend this notion to Renyi common entropy by minimizing the Renyi entropy of the latent variable.
• To efficiently compute common entropy, we propose an iterative algorithm that can be used to discover the trade-off between the entropy of the latent variable and the conditional mutual information of the observed variables.