Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning

Understand

In-context Learning (ICL) has emerged as a powerful capability alongside the development of scaled-up large language models (LLMs).

• By instructing LLMs using few-shot demonstrative examples, ICL enables them to perform a wide range of tasks without updating millions of parameters.
• However, the precise contributions of demonstrations towards improving end-task performance have not been thoroughly investigated in recent analytical studies.
• In this paper, we empirically decompose the overall performance of ICL into three dimensions, label space, format, and discrimination, and we evaluate four general-purpose LLMs across a diverse range of tasks.