Improving the Robustness of Speech Translation

Understand

Although neural machine translation (NMT) has achieved impressive progress recently, it is usually trained on the clean parallel data set and hence cannot work well when the input sentence is the production of the automatic speech recognition (ASR) system due to the enormous errors in the source.

• To solve this problem, we propose a simple but effective method to improve the robustness of NMT in the case of speech translation.
• We simulate the noise existing in the realistic output of the ASR system and inject them into the clean parallel data so that NMT can work under similar word distributions during training and testing.
• Besides, we also incorporate the Chinese Pinyin feature which is easy to get in speech translation to further improve the translation performance.