Learning Discrete Distributions by Dequantization

Understand

Media is generally stored digitally and is therefore discrete.

• Many successful deep distribution models in deep learning learn a density, i.e., the distribution of a continuous random variable.
• Na\"ive optimization on discrete data leads to arbitrarily high likelihoods, and instead, it has become standard practice to add noise to datapoints.
• In this paper, we present a general framework for dequantization that captures existing methods as a special case.