The Roles of English in Evaluating Multilingual Language Models

Understand

Multilingual natural language processing is getting increased attention, with numerous models, benchmarks, and methods being released for many languages.

• English is often used in multilingual evaluation to prompt language models (LMs), mainly to overcome the lack of instruction tuning data in other languages.
• In this position paper, we lay out two roles of English in multilingual LM evaluations: as an interface and as a natural language.
• We argue that these roles have different goals: task performance versus language understanding.