Latent Space Policies for Hierarchical Reinforcement Learning

Understand

We address the problem of learning hierarchical deep neural network policies for reinforcement learning.

• In contrast to methods that explicitly restrict or cripple lower layers of a hierarchy to force them to use higher-level modulating signals, each layer in our framework is trained to directly solve the task, but acquires a range of diverse strategies via a maximum entropy reinforcement learning objective.
• Each layer is also augmented with latent random variables, which are sampled from a prior distribution during the training of that layer.
• The maximum entropy objective causes these latent variables to be incorporated into the layer's policy, and the higher level layer can directly control the behavior of the lower layer through this latent space.