Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks

Understand

While neural machine translation (NMT) models provide improved translation quality in an elegant, end-to-end framework, it is less clear what they learn about language.

• Recent work has started evaluating the quality of vector representations learned by NMT models on morphological and syntactic tasks.
• In this paper, we investigate the representations learned at different layers of NMT encoders.
• We train NMT systems on parallel data and use the trained models to extract features for training a classifier on two tasks: part-of-speech and semantic tagging.