Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback

Understand

Machine translation is a natural candidate problem for reinforcement learning from human feedback: users provide quick, dirty ratings on candidate translations to guide a system to improve.

• Yet, current neural machine translation training focuses on expensive human-generated reference translations.
• We describe a reinforcement learning algorithm that improves neural machine translation systems from simulated human feedback.
• Our algorithm combines the advantage actor-critic algorithm (Mnih et al., 2016) with the attention-based neural encoder-decoder architecture (Luong et al., 2015).