Balanced off-policy evaluation in general action spaces

Understand

Estimation of importance sampling weights for off-policy evaluation of contextual bandits often results in imbalance - a mismatch between the desired and the actual distribution of state-action pairs after weighting.

• In this work we present balanced off-policy evaluation (B-OPE), a generic method for estimating weights which minimize this imbalance.
• Estimation of these weights reduces to a binary classification problem regardless of action type.
• We show that minimizing the risk of the classifier implies minimization of imbalance to the desired counterfactual distribution of state-action pairs.