Epistemic Monte Carlo Tree Search

Understand

The AlphaZero/MuZero (A/MZ) family of algorithms has achieved remarkable success across various challenging domains by integrating Monte Carlo Tree Search (MCTS) with learned models.

• Learned models introduce epistemic uncertainty, which is caused by learning from limited data and is useful for exploration in sparse reward environments.
• MCTS does not account for the propagation of this uncertainty however.
• To address this, we introduce Epistemic MCTS (EMCTS): a theoretically motivated approach to account for the epistemic uncertainty in search and harness the search for deep exploration.