Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Training

Understand

We propose gated language experts and curriculum training to enhance multilingual transformer transducer models without requiring language identification (LID) input from users during inference.

• Our method incorporates a gating mechanism and LID loss, enabling transformer experts to learn language-specific information.
• By combining gated transformer experts with shared transformer layers, we construct multilingual transformer blocks and utilize linear experts to effectively regularize the joint network.
• The curriculum training scheme leverages LID to guide the gated experts in improving their respective language performance.