Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

Understand

Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving.

• This makes memory a critical component, yet its management and evolution remain largely underexplored.
• Existing evaluations mostly focus on static conversational settings, where memory is passively retrieved from dialogue to answer queries, overlooking the dynamic ability to accumulate and reuse experience across evolving task streams.
• In real-world environments such as interactive problem assistants or embodied agents, LLMs are required to handle continuous task streams, yet often fail to learn from accumulated interactions, losing valuable contextual insights, a limitation that calls for test-time evolution, where LLMs retrieve, integrate, and update memory continuously during deployment.