What Drives Performance in Multilingual Language Models?

Understand

This study investigates the factors influencing the performance of multilingual large language models (MLLMs) across diverse languages.

• We study 6 MLLMs, including masked language models, autoregressive models, and instruction-tuned LLMs, on the SIB-200 dataset, a topic classification dataset encompassing 204 languages.
• Our analysis considers three scenarios: ALL languages, SEEN languages (present in the model's pretraining data), and UNSEEN languages (not present or documented in the model's pretraining data in any meaningful way).
• We examine the impact of factors such as pretraining data size, general resource availability, language family, and script type on model performance.