DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environments

Understand

Large language model (LLM)-based agents are increasingly applied to complex strategic environments that demand long-horizon reasoning, multi-agent interaction, and decision-making under uncertainty.

• However, common existing benchmarks either assess isolated skills, lack environmental diversity, or rely on broad overall metrics.
• To address these issues, we introduce DSGBench, a more rigorous evaluation platform for strategic decision-making tasks.
• Firstly, it incorporates six complex strategic games which serve as ideal testbeds due to their long-term and multi-dimensional decision-making demands and flexibility in customizing tasks with various difficulty levels and targets.