Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation

Understand

Diffusion probabilistic models (DPMs) have become a popular approach to conditional generation, due to their promising results and support for cross-modal synthesis.

• A key desideratum in conditional synthesis is to achieve high correspondence between the conditioning input and generated output.
• Most existing methods learn such relationships implicitly, by incorporating the prior into the variational lower bound.
• In this work, we take a different route -- we explicitly enhance input-output connections by maximizing their mutual information.