Deep Data Density Estimation through Donsker-Varadhan Representation

Understand

Estimating the data density is one of the challenging problems in deep learning.

• In this paper, we present a simple yet effective method for estimating the data density using a deep neural network and the Donsker-Varadhan variational lower bound on the KL divergence.
• We show that the optimal critic function associated with the Donsker-Varadhan representation on the KL divergence between the data and the uniform distribution can estimate the data density.
• We also present the deep neural network-based modeling and its stochastic learning.