Controlled Text Generation as Continuous Optimization with Multiple Constraints

Understand

As large-scale language model pretraining pushes the state-of-the-art in text generation, recent work has turned to controlling attributes of the text such models generate.

• While modifying the pretrained models via fine-tuning remains the popular approach, it incurs a significant computational cost and can be infeasible due to lack of appropriate data.
• As an alternative, we propose MuCoCO -- a flexible and modular algorithm for controllable inference from pretrained models.
• We formulate the decoding process as an optimization problem which allows for multiple attributes we aim to control to be easily incorporated as differentiable constraints to the optimization.