Deep Q-Learning: Theoretical Insights from an Asymptotic Analysis

Understand

Deep Q-Learning is an important reinforcement learning algorithm, which involves training a deep neural network, called Deep Q-Network (DQN), to approximate the well-known Q-function.

• Although wildly successful under laboratory conditions, serious gaps between theory and practice as well as a lack of formal guarantees prevent its use in the real world.
• Adopting a dynamical systems perspective, we provide a theoretical analysis of a popular version of Deep Q-Learning under realistic and verifiable assumptions.
• More specifically, we prove an important result on the convergence of the algorithm, characterizing the asymptotic behavior of the learning process.