Diffusion bridges vector quantized Variational AutoEncoders

Understand

Vector Quantized-Variational AutoEncoders (VQ-VAE) are generative models based on discrete latent representations of the data, where inputs are mapped to a finite set of learned embeddings.To generate new samples, an autoregressive prior distribution over the discrete states must be trained separately.

• This prior is generally very complex and leads to slow generation.
• In this work, we propose a new model to train the prior and the encoder/decoder networks simultaneously.
• We build a diffusion bridge between a continuous coded vector and a non-informative prior distribution.