Pessimistic Model Selection for Offline Deep Reinforcement Learning

Understand

Deep Reinforcement Learning (DRL) has demonstrated great potentials in solving sequential decision making problems in many applications.

• Despite its promising performance, practical gaps exist when deploying DRL in real-world scenarios.
• One main barrier is the over-fitting issue that leads to poor generalizability of the policy learned by DRL.
• In particular, for offline DRL with observational data, model selection is a challenging task as there is no ground truth available for performance demonstration, in contrast with the online setting with simulated environments.