Learning Task Automata for Reinforcement Learning using Hidden Markov Models

Understand

Training reinforcement learning (RL) agents using scalar reward signals is often infeasible when an environment has sparse and non-Markovian rewards.

• Moreover, handcrafting these reward functions before training is prone to misspecification, especially when the environment's dynamics are only partially known.
• This paper proposes a novel pipeline for learning non-Markovian task specifications as succinct finite-state `task automata' from episodes of agent experience within unknown environments.
• We leverage two key algorithmic insights.