Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts

Understand

Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people.

• We formulate this as Bayesian Reinforcement Learning over latent Markov Decision Processes (MDPs).
• While Bayes-optimality is theoretically the gold standard, existing algorithms do not scale well to continuous state and action spaces.
• Our proposal builds on the following insight: in the absence of uncertainty, each latent MDP is easier to solve.