A Call for Prudent Choice of Subword Merge Operations in Neural Machine Translation

Understand

Most neural machine translation systems are built upon subword units extracted by methods such as Byte-Pair Encoding (BPE) or wordpiece.

• However, the choice of number of merge operations is generally made by following existing recipes.
• In this paper, we conduct a systematic exploration on different numbers of BPE merge operations to understand how it interacts with the model architecture, the strategy to build vocabularies and the language pair.
• Our exploration could provide guidance for selecting proper BPE configurations in the future.