CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment

Understand

Multilingual proficiency presents a significant challenge for large language models (LLMs).

• English-centric models are usually suboptimal in other languages, particularly those that are linguistically distant from English.
• This performance discrepancy mainly stems from the imbalanced distribution of training data across languages during pre-training and instruction tuning stages.
• To address this problem, we propose a novel approach called CrossIn, which utilizes a mixed composition of cross-lingual instruction tuning data.