Cross-modal Contrastive Learning for Speech Translation

Understand

How can we learn unified representations for spoken utterances and their written text? Learning similar representations for semantically similar speech and text is important for speech translation.

• To this end, we propose ConST, a cross-modal contrastive learning method for end-to-end speech-to-text translation.
• We evaluate ConST and a variety of previous baselines on a popular benchmark MuST-C.
• Experiments show that the proposed ConST consistently outperforms the previous methods on, and achieves an average BLEU of 29.4.