CoCon: A Self-Supervised Approach for Controlled Text Generation

Understand

Pretrained Transformer-based language models (LMs) display remarkable natural language generation capabilities.

• With their immense potential, controlling text generation of such LMs is getting attention.
• While there are studies that seek to control high-level attributes (such as sentiment and topic) of generated text, there is still a lack of more precise control over its content at the word- and phrase-level.
• Here, we propose Content-Conditioner (CoCon) to control an LM's output text with a content input, at a fine-grained level.