Zero-shot Text Classification With Generative Language Models

Understand

This work investigates the use of natural language to enable zero-shot model adaptation to new tasks.

• We use text and metadata from social commenting platforms as a source for a simple pretraining task.
• We then provide the language model with natural language descriptions of classification tasks as input and train it to generate the correct answer in natural language via a language modeling objective.
• This allows the model to generalize to new classification tasks without the need for multiple multitask classification heads.