Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models

Understand

Large language models (LLMs) can perform reasoning computations both internally within their latent space and externally by generating explicit token sequences like chains of thought.

• Significant progress in enhancing reasoning abilities has been made by scaling test-time compute.
• However, understanding and quantifying model-internal reasoning abilities - the inferential "leaps" models make between individual token predictions - remains crucial.
• This study introduces a benchmark (n = 4,000 items) designed to quantify model-internal reasoning in different domains.