Incorporating Structural Alignment Biases into an Attentional Neural Translation Model

Understand

Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models.

• However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models.
• In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions.
• We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.