When Does Translation Require Context? A Data-driven, Multilingual Exploration

Understand

Although proper handling of discourse significantly contributes to the quality of machine translation (MT), these improvements are not adequately measured in common translation quality metrics.

• Recent works in context-aware MT attempt to target a small set of discourse phenomena during evaluation, however not in a fully systematic way.
• In this paper, we develop the Multilingual Discourse-Aware (MuDA) benchmark, a series of taggers that identify and evaluate model performance on discourse phenomena in any given dataset.
• The choice of phenomena is inspired by a novel methodology to systematically identify translations requiring context.