Model-Based Bayesian Exploration

Understand

Reinforcement learning systems are often concerned with balancing exploration of untested actions against exploitation of actions that are known to be good.

• The benefit of exploration can be estimated using the classical notion of Value of Information - the expected improvement in future decision quality arising from the information acquired by exploration.
• Estimating this quantity requires an assessment of the agent's uncertainty about its current value estimates for states.
• In this paper we investigate ways of representing and reasoning about this uncertainty in algorithms where the system attempts to learn a model of its environment.