Reward Shaping for Reinforcement Learning with Omega-Regular Objectives

Understand

Recently, successful approaches have been made to exploit good-for-MDPs automata (B\"uchi automata with a restricted form of nondeterminism) for model free reinforcement learning, a class of automata that subsumes good for games automata and the most widespread class of limit deterministic automata.

• The foundation of using these B\"uchi automata is that the B\"uchi condition can, for good-for-MDP automata, be translated to reachability.
• The drawback of this translation is that the rewards are, on average, reaped very late, which requires long episodes during the learning process.
• We devise a new reward shaping approach that overcomes this issue.