Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounders

Understand

Offline reinforcement learning is important in domains such as medicine, economics, and e-commerce where online experimentation is costly, dangerous or unethical, and where the true model is unknown.

• However, most methods assume all covariates used in the behavior policy's action decisions are observed.
• Though this assumption, sequential ignorability/unconfoundedness, likely does not hold in observational data, most of the data that accounts for selection into treatment may be observed, motivating sensitivity analysis.
• We study robust policy evaluation and policy optimization in the presence of sequentially-exogenous unobserved confounders under a sensitivity model.