Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck

Understand

Variational autoencoders (VAEs) are essential tools in end-to-end representation learning.

• However, the sequential text generation common pitfall with VAEs is that the model tends to ignore latent variables with a strong auto-regressive decoder.
• In this paper, we propose a principled approach to alleviate this issue by applying a discretized bottleneck to enforce an implicit latent feature matching in a more compact latent space.
• We impose a shared discrete latent space where each input is learned to choose a combination of latent atoms as a regularized latent representation.