Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation

Understand

In this work, we propose to apply trust region optimization to deep reinforcement learning using a recently proposed Kronecker-factored approximation to the curvature.

• We extend the framework of natural policy gradient and propose to optimize both the actor and the critic using Kronecker-factored approximate curvature (K-FAC) with trust region; hence we call our method Actor Critic using Kronecker-Factored Trust Region (ACKTR).
• To the best of our knowledge, this is the first scalable trust region natural gradient method for actor-critic methods.
• It is also a method that learns non-trivial tasks in continuous control as well as discrete control policies directly from raw pixel inputs.