Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL

Understand

We present a novel Diffusion Offline Multi-agent Model (DOM2) for offline Multi-Agent Reinforcement Learning (MARL).

• Different from existing algorithms that rely mainly on conservatism in policy design, DOM2 enhances policy expressiveness and diversity based on diffusion model.
• Specifically, we incorporate a diffusion model into the policy network and propose a trajectory-based data-reweighting scheme in training.
• These key ingredients significantly improve algorithm robustness against environment changes and achieve significant improvements in performance, generalization and data-efficiency.