Stochastic Variance Reduction for Deep Q-learning

Understand

Recent advances in deep reinforcement learning have achieved human-level performance on a variety of real-world applications.

• However, the current algorithms still suffer from poor gradient estimation with excessive variance, resulting in unstable training and poor sample efficiency.
• In our paper, we proposed an innovative optimization strategy by utilizing stochastic variance reduced gradient (SVRG) techniques.
• With extensive experiments on Atari domain, our method outperforms the deep q-learning baselines on 18 out of 20 games.