Estimating the Mutual Information between two Discrete, Asymmetric Variables with Limited Samples

Understand

Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields.

• The established measure for quantifying such relations is the mutual information.
• However, estimating mutual information from limited samples is a challenging task.
• Since the mutual information is the difference of two entropies, the existing Bayesian estimators of entropy may be used to estimate information.