Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

Understand

Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks.

• However, task-level expert selection is often too coarse-grained, since different instances may require different expertise.
• To address this, we propose Skill-MoE, a symbolic, skill-based, and gradient-free Mixture-of-Experts framework for instance-level expert selection.
• Skill-MoE infers skills (e.g., algebra in mathematics) from each query, selects experts based on skill relevance, and lets each expert generate its own reasoning.