Bayesian Inference in Monte-Carlo Tree Search

Understand

Monte-Carlo Tree Search (MCTS) methods are drawing great interest after yielding breakthrough results in computer Go.

• This paper proposes a Bayesian approach to MCTS that is inspired by distributionfree approaches such as UCT [13], yet significantly differs in important respects.
• The Bayesian framework allows potentially much more accurate (Bayes-optimal) estimation of node values and node uncertainties from a limited number of simulation trials.
• We further propose propagating inference in the tree via fast analytic Gaussian approximation methods: this can make the overhead of Bayesian inference manageable in domains such as Go, while preserving high accuracy of expected-value estimates.