Critic PI2: Master Continuous Planning via Policy Improvement with Path Integrals and Deep Actor-Critic Reinforcement Learning

Understand

Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence.

• Tree-based planning methods from AlphaGo to Muzero have enjoyed huge success in discrete domains, such as chess and Go.
• Unfortunately, in real-world applications like robot control and inverted pendulum, whose action space is normally continuous, those tree-based planning techniques will be struggling.
• To address those limitations, in this paper, we present a novel model-based reinforcement learning frameworks called Critic PI2, which combines the benefits from trajectory optimization, deep actor-critic learning, and model-based reinforcement learning.