Focused Prefix Tuning for Controllable Text Generation

Understand

In a controllable text generation dataset, there exist unannotated attributes that could provide irrelevant learning signals to models that use it for training and thus degrade their performance.

• We propose focused prefix tuning(FPT) to mitigate the problem and to enable the control to focus on the desired attribute.
• Experimental results show that FPT can achieve better control accuracy and text fluency than baseline models in single-attribute control tasks.
• In multi-attribute control tasks, FPT achieves comparable control accuracy with the state-of-the-art approach while keeping the flexibility to control new attributes without retraining existing models.