Learning to Segment Inputs for NMT Favors Character-Level Processing

Understand

Most modern neural machine translation (NMT) systems rely on presegmented inputs.

• Segmentation granularity importantly determines the input and output sequence lengths, hence the modeling depth, and source and target vocabularies, which in turn determine model size, computational costs of softmax normalization, and handling of out-of-vocabulary words.
• However, the current practice is to use static, heuristic-based segmentations that are fixed before NMT training.
• This begs the question whether the chosen segmentation is optimal for the translation task.