An Information-Theoretic Optimality Principle for Deep Reinforcement Learning

Understand

We methodologically address the problem of Q-value overestimation in deep reinforcement learning to handle high-dimensional state spaces efficiently.

• By adapting concepts from information theory, we introduce an intrinsic penalty signal encouraging reduced Q-value estimates.
• The resultant algorithm encompasses a wide range of learning outcomes containing deep Q-networks as a special case.
• Different learning outcomes can be demonstrated by tuning a Lagrange multiplier accordingly.