MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning

Understand

LLM-based search agents often concatenate the full interaction history into the context, producing long and noisy inputs, and increasing compute cost and GPU memory overhead.

• To address this issue, we propose MemSearcher, an agent framework that maintains a compact memory during multi-turn interactions, retaining only question-relevant information and thereby keeping the context length stable across turns.
• Training MemSearcher is challenging because each trajectory spans multiple turns under different LLM contexts, making each turn an independent optimization target in reinforcement learning.
• We introduce multi-context GRPO, which propagates trajectory-level advantages to all turns for end-to-end optimization.