GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain

Understand

Deep neural language models have set new breakthroughs in many tasks of Natural Language Processing (NLP).

• Recent work has shown that deep transformer language models (pretrained on large amounts of texts) can achieve high levels of task-specific few-shot performance comparable to state-of-the-art models.
• However, the ability of these large language models in few-shot transfer learning has not yet been explored in the biomedical domain.
• We investigated the performance of two powerful transformer language models, i.e.