A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning

Understand

We study what actually works and what doesn't for training large language models as agents via multi-turn reinforcement learning.

• Despite rapid progress, existing frameworks and definitions are fragmented, and there is no systematic formulation or analysis of which design choices matter across tasks.
• We address this gap by first breaking down the design space into three inter-related pillars -- environment, reward, and policy -- and empirically derive a recipe for training LLM agents in situated textual domains.
• In particular, we test TextWorld and ALFWorld, popular domains for testing situated embodied reasoning, as well as SWE-Gym for more software engineering style tasks.