The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation

Understand

The past year has witnessed rapid advances in sequence-to-sequence (seq2seq) modeling for Machine Translation (MT).

• The classic RNN-based approaches to MT were first out-performed by the convolutional seq2seq model, which was then out-performed by the more recent Transformer model.
• Each of these new approaches consists of a fundamental architecture accompanied by a set of modeling and training techniques that are in principle applicable to other seq2seq architectures.
• In this paper, we tease apart the new architectures and their accompanying techniques in two ways.