Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning

Understand

Reasoning large language models (LLMs) heavily rely on scaling test-time compute to perform complex reasoning tasks by generating extensive "thinking" chains.

• While demonstrating impressive results, this approach incurs significant computational costs and inference time.
• In this work, we challenge the assumption that long thinking chains results in better reasoning capabilities.
• We first demonstrate that shorter reasoning chains within individual questions are significantly more likely to yield correct answers - up to 34.5% more accurate than the longest chain sampled for the same question.