EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems

Understand

As large language models (LLMs) continue to advance in programming tasks, LLM-driven coding systems have evolved from one-shot code generation into complex systems capable of iterative improvement during inference.

• However, existing code benchmarks primarily emphasize static correctness and implicitly assume fixed model capability during inference.
• As a result, they do not capture inference-time self-evolution, such as whether accuracy and efficiency improve as an agent iteratively refines its solutions.
• They also provide limited accounting of resource costs and rarely calibrate model performance against that of human programmers.