Neural Machine Translation with Byte-Level Subwords

Understand

Almost all existing machine translation models are built on top of character-based vocabularies: characters, subwords or words.

• Rare characters from noisy text or character-rich languages such as Japanese and Chinese however can unnecessarily take up vocabulary slots and limit its compactness.
• Representing text at the level of bytes and using the 256 byte set as vocabulary is a potential solution to this issue.
• High computational cost has however prevented it from being widely deployed or used in practice.