Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skills

Understand

Mutual information-based reinforcement learning (RL) has been proposed as a promising framework for retrieving complex skills autonomously without a task-oriented reward function through mutual information (MI) maximization or variational empowerment.

• However, learning complex skills is still challenging, due to the fact that the order of training skills can largely affect sample efficiency.
• Inspired by this, we recast variational empowerment as curriculum learning in goal-conditioned RL with an intrinsic reward function, which we name Variational Curriculum RL (VCRL).
• From this perspective, we propose a novel approach to unsupervised skill discovery based on information theory, called Value Uncertainty Variational Curriculum (VUVC).