The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal

Understand

Obtaining meaningful quantitative descriptions of the statistical dependence within multivariate systems is a difficult open problem.

• Recently, the Partial Information Decomposition (PID) was proposed to decompose mutual information (MI) about a target variable into components which are redundant, unique and synergistic within different subsets of predictor variables.
• Here, we propose to apply the elegant formalism of the PID to multivariate entropy, resulting in a Partial Entropy Decomposition (PED).
• We implement the PED with an entropy redundancy measure based on pointwise common surprisal; a natural definition which is closely related to the definition of MI.