Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks

Understand

Foundation models or pre-trained models have substantially improved the performance of various language, vision, and vision-language understanding tasks.

• However, existing foundation models can only perform the best in one type of tasks, namely language, vision, or vision-language.
• It is still an open question whether it is possible to construct a foundation model performing the best for all the understanding tasks, which we call a general foundation model.
• In this paper, we propose a new general foundation model, X-FM (the X-Foundation Model).