Off-Policy Actor-Critic

Understand

This paper presents the first actor-critic algorithm for off-policy reinforcement learning.

• Our algorithm is online and incremental, and its per-time-step complexity scales linearly with the number of learned weights.
• Previous work on actor-critic algorithms is limited to the on-policy setting and does not take advantage of the recent advances in off-policy gradient temporal-difference learning.
• Off-policy techniques, such as Greedy-GQ, enable a target policy to be learned while following and obtaining data from another (behavior) policy.