A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets

Understand

Mutual Information (MI) is a fundamental metric for quantifying dependency between two random variables.

• When we can access only the samples, but not the underlying distribution functions, we can evaluate MI using sample-based estimators.
• Assessment of such MI estimators, however, has almost always relied on analytical datasets including Gaussian multivariates.
• Such datasets allow analytical calculations of the true MI values, but they are limited in that they do not reflect the complexities of real-world datasets.