Deep Recurrent Q-Learning for Partially Observable MDPs

Understand

Deep Reinforcement Learning has yielded proficient controllers for complex tasks.

• However, these controllers have limited memory and rely on being able to perceive the complete game screen at each decision point.
• To address these shortcomings, this article investigates the effects of adding recurrency to a Deep Q-Network (DQN) by replacing the first post-convolutional fully-connected layer with a recurrent LSTM.
• The resulting \textit{Deep Recurrent Q-Network} (DRQN), although capable of seeing only a single frame at each timestep, successfully integrates information through time and replicates DQN's performance on standard Atari games and partially observed equivalents featuring flickering game screens.