Non-autoregressive Transformer by Position Learning

Understand

Non-autoregressive models are promising on various text generation tasks.

• Previous work hardly considers to explicitly model the positions of generated words.
• However, position modeling is an essential problem in non-autoregressive text generation.
• In this study, we propose PNAT, which incorporates positions as a latent variable into the text generative process.