Improved Contrastive Divergence Training of Energy Based Models

Understand

Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability.

• We propose an adaptation to improve contrastive divergence training by scrutinizing a gradient term that is difficult to calculate and is often left out for convenience.
• We show that this gradient term is numerically significant and in practice is important to avoid training instabilities, while being tractable to estimate.
• We further highlight how data augmentation and multi-scale processing can be used to improve model robustness and generation quality.