The Impact of Language Adapters in Cross-Lingual Transfer for NLU

Understand

Modular deep learning has been proposed for the efficient adaption of pre-trained models to new tasks, domains and languages.

• In particular, combining language adapters with task adapters has shown potential where no supervised data exists for a language.
• In this paper, we explore the role of language adapters in zero-shot cross-lingual transfer for natural language understanding (NLU) benchmarks.
• We study the effect of including a target-language adapter in detailed ablation studies with two multilingual models and three multilingual datasets.