When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification

Understand

Text classification tends to be difficult when data are deficient or when it is required to adapt to unseen classes.

• In such challenging scenarios, recent studies have often used meta-learning to simulate the few-shot task, thus negating implicit common linguistic features across tasks.
• This paper addresses such problems using meta-learning and unsupervised language models.
• Our approach is based on the insight that having a good generalization from a few examples relies on both a generic model initialization and an effective strategy for adapting this model to newly arising tasks.