SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models

Understand

There is a growing trend of teaching large language models (LLMs) to solve mathematical problems through coding.

• Existing studies primarily focus on prompting powerful, closed-source models to generate seed training data followed by in-domain data augmentation, equipping LLMs with considerable capabilities for code-aided mathematical reasoning.
• However, continually training these models on augmented data derived from a few datasets such as GSM8K may impair their generalization abilities and restrict their effectiveness to a narrow range of question types.
• Conversely, the potential of improving such LLMs by leveraging large-scale, expert-written, diverse math question-answer pairs remains unexplored.