Adversarially Guided Actor-Critic

Understand

Despite definite success in deep reinforcement learning problems, actor-critic algorithms are still confronted with sample inefficiency in complex environments, particularly in tasks where efficient exploration is a bottleneck.

• These methods consider a policy (the actor) and a value function (the critic) whose respective losses are built using different motivations and approaches.
• This paper introduces a third protagonist: the adversary.
• While the adversary mimics the actor by minimizing the KL-divergence between their respective action distributions, the actor, in addition to learning to solve the task, tries to differentiate itself from the adversary predictions.