How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Model

Understand

Energy-based models parameterize the unnormalized log-probability of data samples, but there is a lack of guidance on how to construct the "energy".

• In this paper, we propose a Denoising-EBM which decomposes the image energy into "semantic energy" and "texture energy".
• We define the "semantic energy" in the latent space of DAE to model the high-level representations, and define the pixel-level reconstruction error for denoising as "texture energy".
• Inspired by score-based model, our model utilizes multi-scale noisy samples for maximum-likelihood training and it outputs a vector instead of a scalar for exploring a larger set of functions during optimization.