How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning

Understand

Multilingual large language models (MLLMs) are jointly trained on data from many different languages such that representation of individual languages can benefit from other languages' data.

• Impressive performance on zero-shot cross-lingual transfer shows that these models are capable of exploiting data from other languages.
• Yet, it remains unclear to what extent, and under which conditions, languages rely on each other's data.
• In this study, we use TracIn (Pruthi et al., 2020), a training data attribution (TDA) method, to retrieve the most influential training samples seen during multilingual fine-tuning for a particular test language.