Do Massively Pretrained Language Models Make Better Storytellers?

Understand

Large neural language models trained on massive amounts of text have emerged as a formidable strategy for Natural Language Understanding tasks.

• However, the strength of these models as Natural Language Generators is less clear.
• Though anecdotal evidence suggests that these models generate better quality text, there has been no detailed study characterizing their generation abilities.
• In this work, we compare the performance of an extensively pretrained model, OpenAI GPT2-117 (Radford et al., 2019), to a state-of-the-art neural story generation model (Fan et al., 2018).