Bregman divergence as general framework to estimate unnormalized statistical models

Understand

We show that the Bregman divergence provides a rich framework to estimate unnormalized statistical models for continuous or discrete random variables, that is, models which do not integrate or sum to one, respectively.

• We prove that recent estimation methods such as noise-contrastive estimation, ratio matching, and score matching belong to the proposed framework, and explain their interconnection based on supervised learning.
• Further, we discuss the role of boosting in unsupervised learning.