Neural Methods for Point-wise Dependency Estimation

Understand

Since its inception, the neural estimation of mutual information (MI) has demonstrated the empirical success of modeling expected dependency between high-dimensional random variables.

• However, MI is an aggregate statistic and cannot be used to measure point-wise dependency between different events.
• In this work, instead of estimating the expected dependency, we focus on estimating point-wise dependency (PD), which quantitatively measures how likely two outcomes co-occur.
• We show that we can naturally obtain PD when we are optimizing MI neural variational bounds.