Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation

Understand

Instruction tuning has shown great promise in improving the performance of large language models.

• However, research on multilingual instruction tuning has been limited due to the scarcity of high-quality instruction-response datasets across different languages.
• To bridge this gap, we present Bactrian-X, a comprehensive multilingual parallel dataset of 3.4 million instruction-response pairs across 52 languages.
• Leveraging this dataset, we train a set of adapters using low-rank adaptation (LoRA), which are lightweight components that seamlessly integrate with large language models.