Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards

Understand

Hierarchical Reinforcement Learning (HRL) is a promising approach to solving long-horizon problems with sparse and delayed rewards.

• Many existing HRL algorithms either use pre-trained low-level skills that are unadaptable, or require domain-specific information to define low-level rewards.
• In this paper, we aim to adapt low-level skills to downstream tasks while maintaining the generality of reward design.
• We propose an HRL framework which sets auxiliary rewards for low-level skill training based on the advantage function of the high-level policy.