EuroBERT: Scaling Multilingual Encoders for European Languages

Understand

General-purpose multilingual vector representations, used in retrieval, regression and classification, are traditionally obtained from bidirectional encoder models.

• Despite their wide applicability, encoders have been recently overshadowed by advances in generative decoder-only models.
• However, many innovations driving this progress are not inherently tied to decoders.
• In this paper, we revisit the development of multilingual encoders through the lens of these advances, and introduce EuroBERT, a family of multilingual encoders covering European and widely spoken global languages.