DiffusER: Discrete Diffusion via Edit-based Reconstruction

Understand

In text generation, models that generate text from scratch one token at a time are currently the dominant paradigm.

• Despite being performant, these models lack the ability to revise existing text, which limits their usability in many practical scenarios.
• We look to address this, with DiffusER (Diffusion via Edit-based Reconstruction), a new edit-based generative model for text based on denoising diffusion models -- a class of models that use a Markov chain of denoising steps to incrementally generate data.
• DiffusER is not only a strong generative model in general, rivalling autoregressive models on several tasks spanning machine translation, summarization, and style transfer; it can also perform other varieties of generation that standard autoregressive models are not well-suited for.