Dual RL: Unification and New Methods for Reinforcement and Imitation Learning

Understand

The goal of reinforcement learning (RL) is to find a policy that maximizes the expected cumulative return.

• It has been shown that this objective can be represented as an optimization problem of state-action visitation distribution under linear constraints.
• The dual problem of this formulation, which we refer to as dual RL, is unconstrained and easier to optimize.
• In this work, we first cast several state-of-the-art offline RL and offline imitation learning (IL) algorithms as instances of dual RL approaches with shared structures.