Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model

Understand

The recently released NLLB-200 is a set of multilingual Neural Machine Translation models that cover 202 languages.

• The largest model is based on a Mixture of Experts architecture and achieves SoTA results across many language pairs.
• It contains 54.5B parameters and requires at least four 32GB GPUs just for inference.
• In this work, we propose a pruning method that enables the removal of up to 80% of experts without further finetuning and with a negligible loss in translation quality, which makes it feasible to run the model on a single 32GB GPU.