Conditional Noise-Contrastive Estimation of Unnormalised Models

Understand

Many parametric statistical models are not properly normalised and only specified up to an intractable partition function, which renders parameter estimation difficult.

• Examples of unnormalised models are Gibbs distributions, Markov random fields, and neural network models in unsupervised deep learning.
• In previous work, the estimation principle called noise-contrastive estimation (NCE) was introduced where unnormalised models are estimated by learning to distinguish between data and auxiliary noise.
• An open question is how to best choose the auxiliary noise distribution.