Effective Approaches to Attention-based Neural Machine Translation

Understand

An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation.

• However, there has been little work exploring useful architectures for attention-based NMT.
• This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time.
• We demonstrate the effectiveness of both approaches over the WMT translation tasks between English and German in both directions.