Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps

Understand

We investigate the role of various demonstration components in the in-context learning (ICL) performance of large language models (LLMs).

• Specifically, we explore the impacts of ground-truth labels, input distribution, and complementary explanations, particularly when these are altered or perturbed.
• We build on previous work, which offers mixed findings on how these elements influence ICL.
• To probe these questions, we employ explainable NLP (XNLP) methods and utilize saliency maps of contrastive demonstrations for both qualitative and quantitative analysis.