OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models

Understand

Generalist models, which are capable of performing diverse multi-modal tasks in a task-agnostic way within a single model, have been explored recently.

• Being, hopefully, an alternative to approaching general-purpose AI, existing generalist models are still at an early stage, where modality and task coverage is limited.
• To empower multi-modal task-scaling and speed up this line of research, we release a generalist model learning system, OFASys, built on top of a declarative task interface named multi-modal instruction.
• At the core of OFASys is the idea of decoupling multi-modal task representations from the underlying model implementations.