Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning

Understand

Reinforcement learning (RL) methods usually treat reward functions as black boxes.

• As such, these methods must extensively interact with the environment in order to discover rewards and optimal policies.
• In most RL applications, however, users have to program the reward function and, hence, there is the opportunity to make the reward function visible -- to show the reward function's code to the RL agent so it can exploit the function's internal structure to learn optimal policies in a more sample efficient manner.
• In this paper, we show how to accomplish this idea in two steps.