A Reparameterized Discrete Diffusion Model for Text Generation

Understand

This work studies discrete diffusion probabilistic models with applications to natural language generation.

• We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models.
• The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques.
• We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models.