Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Models

Understand

Multi-agent interactions have long fueled innovation, from natural predator-prey dynamics to the space race.

• Self-play (SP) algorithms try to harness these dynamics by pitting agents against ever-improving opponents, thereby creating an implicit curriculum toward learning high-quality solutions.
• However, SP often fails to produce diverse solutions and can get stuck in locally optimal behaviors.
• We introduce Foundation-Model Self-Play (FMSP), a new direction that leverages the code-generation capabilities and vast knowledge of foundation models (FMs) to overcome these challenges by leaping across local optima in policy space.