On Using Very Large Target Vocabulary for Neural Machine Translation

Understand

Neural machine translation, a recently proposed approach to machine translation based purely on neural networks, has shown promising results compared to the existing approaches such as phrase-based statistical machine translation.

• Despite its recent success, neural machine translation has its limitation in handling a larger vocabulary, as training complexity as well as decoding complexity increase proportionally to the number of target words.
• In this paper, we propose a method that allows us to use a very large target vocabulary without increasing training complexity, based on importance sampling.
• We show that decoding can be efficiently done even with the model having a very large target vocabulary by selecting only a small subset of the whole target vocabulary.