CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing

Understand

Sequential reasoning in agent systems has been significantly advanced by large language models (LLMs), yet existing approaches face limitations.

• Reflection-driven reasoning relies solely on knowledge in pretrained models, limiting performance in novel scenarios, while experience-assisted reasoning often depends on external experiences and lacks clear principles for selecting representative experiences.
• We address these limitations by proposing CoPS (Cross-Task Experience Sharing), a generalizable algorithm that enhances sequential reasoning by cross-task experience sharing and selection.
• In detail, CoPS leverages agents' experiences on previous tasks, selecting distribution-matched experiences via a provable pessimism-based strategy to maximize utility while minimizing risks from distribution shifts.