Evolution of Q Values for Deep Q Learning in Stable Baselines

Understand

We investigate the evolution of the Q values for the implementation of Deep Q Learning (DQL) in the Stable Baselines library.

• Stable Baselines incorporates the latest Reinforcement Learning techniques and achieves superhuman performance in many game environments.
• However, for some simple non-game environments, the DQL in Stable Baselines can struggle to find the correct actions.
• In this paper we aim to understand the types of environment where this suboptimal behavior can happen, and also investigate the corresponding evolution of the Q values for individual states.