Actor-Critic Policy Optimization in Partially Observable Multiagent Environments

Understand

Optimization of parameterized policies for reinforcement learning (RL) is an important and challenging problem in artificial intelligence.

• Among the most common approaches are algorithms based on gradient ascent of a score function representing discounted return.
• In this paper, we examine the role of these policy gradient and actor-critic algorithms in partially-observable multiagent environments.
• We show several candidate policy update rules and relate them to a foundation of regret minimization and multiagent learning techniques for the one-shot and tabular cases, leading to previously unknown convergence guarantees.