Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering

Understand

This paper introduces an automated skill acquisition framework in reinforcement learning which involves identifying a hierarchical description of the given task in terms of abstract states and extended actions between abstract states.

• Identifying such structures present in the task provides ways to simplify and speed up reinforcement learning algorithms.
• These structures also help to generalize such algorithms over multiple tasks without relearning policies from scratch.
• We use ideas from dynamical systems to find metastable regions in the state space and associate them with abstract states.