TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning

Understand

Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL.

• On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori.
• However, in complex environments where transition models need to be learned from data, the deficiencies of learned models have limited their utility for planning.
• To address these challenges, we propose TreeQN, a differentiable, recursive, tree-structured model that serves as a drop-in replacement for any value function network in deep RL with discrete actions.