Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

Understand

Recent work has demonstrated that pre-trained language models (PLMs) are zero-shot learners.

• However, most existing zero-shot methods involve heavy human engineering or complicated self-training pipelines, hindering their application to new situations.
• In this work, we show that zero-shot text classification can be improved simply by clustering texts in the embedding spaces of PLMs.
• Specifically, we fit the unlabeled texts with a Bayesian Gaussian Mixture Model after initializing cluster positions and shapes using class names.