Best Response Shaping

Understand

We investigate the challenge of multi-agent deep reinforcement learning in partially competitive environments, where traditional methods struggle to foster reciprocity-based cooperation.

• LOLA and POLA agents learn reciprocity-based cooperative policies by differentiation through a few look-ahead optimization steps of their opponent.
• However, there is a key limitation in these techniques.
• Because they consider a few optimization steps, a learning opponent that takes many steps to optimize its return may exploit them.