SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learning

Understand

The performance of offline reinforcement learning (RL) suffers from the limited size and quality of static datasets.

• Model-based offline RL addresses this issue by generating synthetic samples through a dynamics model to enhance overall performance.
• To evaluate the reliability of the generated samples, uncertainty estimation methods are often employed.
• However, model ensemble, the most commonly used uncertainty estimation method, is not always the best choice.