AccMER: Accelerating Multi-Agent Experience Replay with Cache Locality-aware Prioritization

Understand

Multi-Agent Experience Replay (MER) is a key component of off-policy reinforcement learning~(RL) algorithms.

• By remembering and reusing experiences from the past, experience replay significantly improves the stability of RL algorithms and their learning efficiency.
• In many scenarios, multiple agents interact in a shared environment during online training under centralized training and decentralized execution~(CTDE) paradigm.
• Current multi-agent reinforcement learning~(MARL) algorithms consider experience replay with uniform sampling or based on priority weights to improve transition data sample efficiency in the sampling phase.