Few-Shot Text Generation with Pattern-Exploiting Training

Understand

Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.

• Moreover, when combined with regular learning from examples, this idea yields impressive few-shot results for a wide range of text classification tasks.
• It is also a promising direction to improve data efficiency in generative settings, but there are several challenges to using a combination of task descriptions and example-based learning for text generation.
• In particular, it is crucial to find task descriptions that are easy to understand for the pretrained model and to ensure that it actually makes good use of them; furthermore, effective measures against overfitting have to be implemented.