Linear Correlation in LM's Compositional Generalization and Hallucination

Understand

The generalization of language models (LMs) is undergoing active debates, contrasting their potential for general intelligence with their struggles with basic knowledge composition (e.g., reverse/transition curse).

• This paper uncovers the phenomenon of linear correlations in LMs during knowledge composition.
• For explanation, there exists a linear transformation between certain related knowledge that maps the next token prediction logits from one prompt to another, e.g., "X lives in the city of" $\rightarrow$ "X lives in the country of" for every given X.
• This mirrors the linearity in human knowledge composition, such as Paris $\rightarrow$ France.