k-Sliced Mutual Information: A Quantitative Study of Scalability with Dimension

Understand

Sliced mutual information (SMI) is defined as an average of mutual information (MI) terms between one-dimensional random projections of the random variables.

• It serves as a surrogate measure of dependence to classic MI that preserves many of its properties but is more scalable to high dimensions.
• However, a quantitative characterization of how SMI itself and estimation rates thereof depend on the ambient dimension, which is crucial to the understanding of scalability, remain obscure.
• This work provides a multifaceted account of the dependence of SMI on dimension, under a broader framework termed $k$-SMI, which considers projections to $k$-dimensional subspaces.