How do Language Models Bind Entities in Context?

Understand

To correctly use in-context information, language models (LMs) must bind entities to their attributes.

• For example, given a context describing a "green square" and a "blue circle", LMs must bind the shapes to their respective colors.
• We analyze LM representations and identify the binding ID mechanism: a general mechanism for solving the binding problem, which we observe in every sufficiently large model from the Pythia and LLaMA families.
• Using causal interventions, we show that LMs' internal activations represent binding information by attaching binding ID vectors to corresponding entities and attributes.