English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings

Understand

Universal cross-lingual sentence embeddings map semantically similar cross-lingual sentences into a shared embedding space.

• Aligning cross-lingual sentence embeddings usually requires supervised cross-lingual parallel sentences.
• In this work, we propose mSimCSE, which extends SimCSE to multilingual settings and reveal that contrastive learning on English data can surprisingly learn high-quality universal cross-lingual sentence embeddings without any parallel data.
• In unsupervised and weakly supervised settings, mSimCSE significantly improves previous sentence embedding methods on cross-lingual retrieval and multilingual STS tasks.