WACO: Word-Aligned Contrastive Learning for Speech Translation

Understand

End-to-end Speech Translation (E2E ST) aims to directly translate source speech into target text.

• Existing ST methods perform poorly when only extremely small speech-text data are available for training.
• We observe that an ST model's performance closely correlates with its embedding similarity between speech and source transcript.
• In this paper, we propose Word-Aligned COntrastive learning (WACO), a simple and effective method for extremely low-resource speech-to-text translation.