Why Global Performance is a Poor Metric for Verifying Convergence of Multi-agent Learning

Understand

Experimental verification has been the method of choice for verifying the stability of a multi-agent reinforcement learning (MARL) algorithm as the number of agents grows and theoretical analysis becomes prohibitively complex.

• For cooperative agents, where the ultimate goal is to optimize some global metric, the stability is usually verified by observing the evolution of the global performance metric over time.
• If the global metric improves and eventually stabilizes, it is considered a reasonable verification of the system's stability.
• The main contribution of this note is establishing the need for better experimental frameworks and measures to assess the stability of large-scale adaptive cooperative systems.