Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs

Understand

To build an artificial neural network like the biological intelligence system, recent works have unified numerous tasks into a generalist model, which can process various tasks with shared parameters and do not have any task-specific modules.

• While generalist models achieve promising results on various benchmarks, they have performance degradation on some tasks compared with task-specialized models.
• In this work, we find that interference among different tasks and modalities is the main factor to this phenomenon.
• To mitigate such interference, we introduce the Conditional Mixture-of-Experts (Conditional MoEs) to generalist models.