Bi-level Actor-Critic for Multi-agent Coordination

Understand

Coordination is one of the essential problems in multi-agent systems.

• Typically multi-agent reinforcement learning (MARL) methods treat agents equally and the goal is to solve the Markov game to an arbitrary Nash equilibrium (NE) when multiple equilibra exist, thus lacking a solution for NE selection.
• In this paper, we treat agents \emph{unequally} and consider Stackelberg equilibrium as a potentially better convergence point than Nash equilibrium in terms of Pareto superiority, especially in cooperative environments.
• Under Markov games, we formally define the bi-level reinforcement learning problem in finding Stackelberg equilibrium.