Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models

Understand

Fine-tuning large language models (LLMs) for reasoning tasks using reinforcement learning methods like Group Relative Policy Optimization (GRPO) is computationally expensive.

• To address this, we propose a predictive framework that models training dynamics and helps optimize resource usage.
• Through experiments on Llama and Qwen models (3B 8B), we derive an empirical scaling law based on model size, initial performance, and training progress.
• This law predicts reward trajectories and identifies three consistent training phases: slow start, rapid improvement, and plateau.