Off-Policy Evaluation in Partially Observable Environments

Understand

This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments.

• Off-policy evaluation under partial observability is inherently prone to bias, with risk of arbitrarily large errors.
• We define the problem of off-policy evaluation for Partially Observable Markov Decision Processes (POMDPs) and establish what we believe is the first off-policy evaluation result for POMDPs.
• In addition, we formulate a model in which observed and unobserved variables are decoupled into two dynamic processes, called a Decoupled POMDP.