LLM Probability Concentration: How Alignment Shrinks the Generative Horizon

Understand

Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity.

• What drives this consistency in the generation? We investigate this phenomenon through the lens of probability concentration in the model's output distribution.
• To quantify it, we use the Branching Factor (BF)--the exponentiated length-averaged entropy of the output distribution, interpreted as the effective number of plausible next steps during generation.
• Our empirical analysis reveals two key findings: (1) BF often decreases as generation progresses, suggesting that LLMs become more predictable; a controlled intervention indicates that this decline is largely a task-independent property of autoregressive self-conditioning, distinct from alignment, and can be locally reversed by unexpected context.