Counterfactual Learning from Bandit Feedback under Deterministic Logging: A Case Study in Statistical Machine Translation

Understand

The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT system.

• A challenge arises by the fact that risk-averse commercial SMT systems deterministically log the most probable translation.
• The lack of sufficient exploration of the SMT output space seemingly contradicts the theoretical requirements for counterfactual learning.
• We show that counterfactual learning from deterministic bandit logs is possible nevertheless by smoothing out deterministic components in learning.