Knowledge Distillation for Quality Estimation

Understand

Quality Estimation (QE) is the task of automatically predicting Machine Translation quality in the absence of reference translations, making it applicable in real-time settings, such as translating online social media conversations.

• Recent success in QE stems from the use of multilingual pre-trained representations, where very large models lead to impressive results.
• However, the inference time, disk and memory requirements of such models do not allow for wide usage in the real world.
• Models trained on distilled pre-trained representations remain prohibitively large for many usage scenarios.