Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation

Understand

Large Language Models (LLMs) exhibit remarkable text classification capabilities, excelling in zero- and few-shot learning (ZSL and FSL) scenarios.

• However, since they are trained on different datasets, performance varies widely across tasks between those models.
• Recent studies emphasize the importance of considering human label variation in data annotation.
• However, how this human label variation also applies to LLMs remains unexplored.