MOSA: Mixtures of Simple Adapters Outperform Monolithic Approaches in LLM-based Multilingual ASR

Understand

LLM-based ASR overcomes multilingual data scarcity by projecting speech representations into the LLM space to leverage its robust semantic and reasoning capabilities.

• However, while previous approaches typically enhance performance by scaling data or model parameters, a single projector often struggles to effectively align representations across different languages.
• In this work, we propose an MoE-based projector named MOSA (Mixture of Simple Adapters).
• By aggregating multiple simple adapters, this architecture enables different experts to specialize in learning either language-shared or language-specific knowledge.