AdaPrompt: Adaptive Model Training for Prompt-based NLP

Understand

Prompt-based learning, with its capability to tackle zero-shot and few-shot NLP tasks, has gained much attention in community.

• The main idea is to bridge the gap between NLP downstream tasks and language modeling (LM), by mapping these tasks into natural language prompts, which are then filled by pre-trained language models (PLMs).
• However, for prompt learning, there are still two salient gaps between NLP tasks and pretraining.
• First, prompt information is not necessarily sufficiently present during LM pretraining.