Hierarchical Imitation Learning with Vector Quantized Models

Understand

The ability to plan actions on multiple levels of abstraction enables intelligent agents to solve complex tasks effectively.

• However, learning the models for both low and high-level planning from demonstrations has proven challenging, especially with higher-dimensional inputs.
• To address this issue, we propose to use reinforcement learning to identify subgoals in expert trajectories by associating the magnitude of the rewards with the predictability of low-level actions given the state and the chosen subgoal.
• We build a vector-quantized generative model for the identified subgoals to perform subgoal-level planning.