EconCausal: A Context-Aware Economic Reasoning Benchmark for Large Language Models

Understand

Socio-economic causal effects depend heavily on their institutional and environmental contexts.

• The same intervention can produce different, even opposite, effects across regulatory regimes, market conditions, time periods, or populations.
• This poses a challenge for large language models (LLMs) in decision-support roles: can they infer the direction of a causal effect under a specified context, and revise that judgment when the context changes? To address this, we introduce EconCausal, a large-scale benchmark of 10,490 context-annotated causal triplets extracted from 2,595 high-quality empirical studies in top-tier economics and finance journals, constructed through a rigorous four-stage pipeline with multi-run consensus, context refinement, and multi-critic filtering.
• Across models, LLMs often fail to condition their predictions on context.