Towards Tractable Optimism in Model-Based Reinforcement Learning

Understand

The principle of optimism in the face of uncertainty is prevalent throughout sequential decision making problems such as multi-armed bandits and reinforcement learning (RL).

• To be successful, an optimistic RL algorithm must over-estimate the true value function (optimism) but not by so much that it is inaccurate (estimation error).
• In the tabular setting, many state-of-the-art methods produce the required optimism through approaches which are intractable when scaling to deep RL.
• We re-interpret these scalable optimistic model-based algorithms as solving a tractable noise augmented MDP.