An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models

Understand

The capacity and effectiveness of pre-trained multilingual models (MLMs) for zero-shot cross-lingual transfer is well established.

• However, phenomena of positive or negative transfer, and the effect of language choice still need to be fully understood, especially in the complex setting of massively multilingual LMs.
• We propose an \textit{efficient} method to study transfer language influence in zero-shot performance on another target language.
• Unlike previous work, our approach disentangles downstream tasks from language, using dedicated adapter units.