Zero-Shot Controlled Generation with Encoder-Decoder Transformers

Understand

Controlling neural network-based models for natural language generation (NLG) has broad applications in numerous areas such as machine translation, document summarization, and dialog systems.

• Approaches that enable such control in a zero-shot manner would be of great importance as, among other reasons, they remove the need for additional annotated data and training.
• In this work, we propose novel approaches for controlling encoder-decoder transformer-based NLG models in zero-shot.
• This is done by introducing three control knobs, namely, attention biasing, decoder mixing, and context augmentation, that are applied to these models at generation time.