A Theory of Mind Approach as Test-Time Mitigation Against Emergent Adversarial Communication

Understand

Multi-Agent Systems (MAS) is the study of multi-agent interactions in a shared environment.

• Communication for cooperation is a fundamental construct for sharing information in partially observable environments.
• Cooperative Multi-Agent Reinforcement Learning (CoMARL) is a learning framework where we learn agent policies either with cooperative mechanisms or policies that exhibit cooperative behavior.
• Explicitly, there are works on learning to communicate messages from CoMARL agents; however, non-cooperative agents, when capable of access a cooperative team's communication channel, have been shown to learn adversarial communication messages, sabotaging the cooperative team's performance particularly when objectives depend on finite resources.