SALTED: A Framework for SAlient Long-Tail Translation Error Detection

Understand

Traditional machine translation (MT) metrics provide an average measure of translation quality that is insensitive to the long tail of behavioral problems in MT.

• Examples include translation of numbers, physical units, dropped content and hallucinations.
• These errors, which occur rarely and unpredictably in Neural Machine Translation (NMT), greatly undermine the reliability of state-of-the-art MT systems.
• Consequently, it is important to have visibility into these problems during model development.