LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion

Understand

Large language models (LLMs) have demonstrated remarkable progress in understanding long-context inputs.

• However, benchmarks for evaluating the long-context reasoning abilities of LLMs fall behind the pace.
• Existing benchmarks often focus on a narrow range of tasks or those that do not demand complex reasoning.
• To address this gap and enable a more comprehensive evaluation of the long-context reasoning capabilities of current LLMs, we propose a new synthetic benchmark, LongReason, which is constructed by synthesizing long-context reasoning questions from a varied set of short-context reasoning questions through context expansion.