CUP: Critic-Guided Policy Reuse

Understand

The ability to reuse previous policies is an important aspect of human intelligence.

• To achieve efficient policy reuse, a Deep Reinforcement Learning (DRL) agent needs to decide when to reuse and which source policies to reuse.
• Previous methods solve this problem by introducing extra components to the underlying algorithm, such as hierarchical high-level policies over source policies, or estimations of source policies' value functions on the target task.
• However, training these components induces either optimization non-stationarity or heavy sampling cost, significantly impairing the effectiveness of transfer.