TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning

Understand

Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning.

• However, existing RL methods like GRPO typically rely on unstructured self-sampling to fit scalar rewards, often producing inefficient rollouts that fail to capture transferable problem-solving strategies.
• To address this limitation, we propose **TemplateRL**, a structured template-guided RL framework that augments policy optimization with explicit template guidance.
• Our approach first constructs a problem-solving template library via MCTS on a small seed set, then seamlessly integrates this high-level structured guidance into RL training.