Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

Understand

As large language models (LLMs) are adopted as a fundamental component of language technologies, it is crucial to accurately characterize their performance.

• Because choices in prompt design can strongly influence model behavior, this design process is critical in effectively using any modern pre-trained generative language model.
• In this work, we focus on LLM sensitivity to a quintessential class of meaning-preserving design choices: prompt formatting.
• We find that several widely used open-source LLMs are extremely sensitive to subtle changes in prompt formatting in few-shot settings, with performance differences of up to 76 accuracy points when evaluated using LLaMA-2-13B.