Differentiable Trust Region Layers for Deep Reinforcement Learning

Understand

Trust region methods are a popular tool in reinforcement learning as they yield robust policy updates in continuous and discrete action spaces.

• However, enforcing such trust regions in deep reinforcement learning is difficult.
• Hence, many approaches, such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO), are based on approximations.
• Due to those approximations, they violate the constraints or fail to find the optimal solution within the trust region.