SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinese

Understand

We introduce SuperCLUE-Math6(SC-Math6), a new benchmark dataset to evaluate the mathematical reasoning abilities of Chinese language models.

• SC-Math6 is designed as an upgraded Chinese version of the GSM8K dataset with enhanced difficulty, diversity, and application scope.
• It consists of over 2000 mathematical word problems requiring multi-step reasoning and providing natural language solutions.
• We propose an innovative scheme to quantify the reasoning capability of large models based on performance over problems with different reasoning steps.