Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning

Understand

We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network.

• The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards.
• To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information.
• In particular, we show that, despite restricting each agent's attention to only its $\kappa$-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in $\kappa$.