WARP: Word-level Adversarial ReProgramming

Understand

Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks.

• A common approach to transfer learning for multiple tasks that maximize parameter sharing trains one or more task-specific layers on top of the language model.
• In this paper, we present an alternative approach based on adversarial reprogramming, which extends earlier work on automatic prompt generation.
• Adversarial reprogramming attempts to learn task-specific word embeddings that, when concatenated to the input text, instruct the language model to solve the specified task.