Neural Text Generation: Past, Present and Beyond

Understand

This paper presents a systematic survey on recent development of neural text generation models.

• Specifically, we start from recurrent neural network language models with the traditional maximum likelihood estimation training scheme and point out its shortcoming for text generation.
• We thus introduce the recently proposed methods for text generation based on reinforcement learning, re-parametrization tricks and generative adversarial nets (GAN) techniques.
• We compare different properties of these models and the corresponding techniques to handle their common problems such as gradient vanishing and generation diversity.