Embodied Agents Meet Personalization: Investigating Challenges and Solutions Through the Lens of Memory Utilization

Understand

LLM-powered embodied agents have shown success on conventional object-rearrangement tasks, but providing personalized assistance that leverages user-specific knowledge from past interactions presents new challenges.

• We investigate these challenges through the lens of agents' memory utilization along two critical dimensions: object semantics (identifying objects based on personal meaning) and user patterns (recalling sequences from behavioral routines).
• To assess these capabilities, we construct MEMENTO, an end-to-end two-stage evaluation framework comprising single-memory and joint-memory tasks.
• Our experiments reveal that current agents can recall simple object semantics but struggle to apply sequential user patterns to planning.