When Thinking Backfires: Mechanistic Insights Into Reasoning-Induced Misalignment

Understand

With the growing accessibility and wide adoption of large language models, concerns about their safety and alignment with human values have become paramount.

• In this paper, we identify a concerning phenomenon: Reasoning-Induced Misalignment (RIM), in which misalignment emerges when reasoning capabilities strengthened-particularly when specific types of reasoning patterns are introduced during inference or training.
• Beyond reporting this vulnerability, we provide the first mechanistic account of its origins.
• Through representation analysis, we discover that specific attention heads facilitate refusal by reducing their attention to CoT tokens, a mechanism that modulates the model's rationalization process during inference.