A Plug-and-Play Method for Controlled Text Generation

Understand

Large pre-trained language models have repeatedly shown their ability to produce fluent text.

• Yet even when starting from a prompt, generation can continue in many plausible directions.
• Current decoding methods with the goal of controlling generation, e.g., to ensure specific words are included, either require additional models or fine-tuning, or work poorly when the task at hand is semantically unconstrained, e.g., story generation.
• In this work, we present a plug-and-play decoding method for controlled language generation that is so simple and intuitive, it can be described in a single sentence: given a topic or keyword, we add a shift to the probability distribution over our vocabulary towards semantically similar words.