Data Efficient Direct Speech-to-Text Translation with Modality Agnostic Meta-Learning

Understand

End-to-end Speech Translation (ST) models have several advantages such as lower latency, smaller model size, and less error compounding over conventional pipelines that combine Automatic Speech Recognition (ASR) and text Machine Translation (MT) models.

• However, collecting large amounts of parallel data for ST task is more difficult compared to the ASR and MT tasks.
• Previous studies have proposed the use of transfer learning approaches to overcome the above difficulty.
• These approaches benefit from weakly supervised training data, such as ASR speech-to-transcript or MT text-to-text translation pairs.