Iterative Empirical Game Solving via Single Policy Best Response

Understand

Policy-Space Response Oracles (PSRO) is a general algorithmic framework for learning policies in multiagent systems by interleaving empirical game analysis with deep reinforcement learning (Deep RL).

• At each iteration, Deep RL is invoked to train a best response to a mixture of opponent policies.
• The repeated application of Deep RL poses an expensive computational burden as we look to apply this algorithm to more complex domains.
• We introduce two variations of PSRO designed to reduce the amount of simulation required during Deep RL training.