Stone Needle: A General Multimodal Large-scale Model Framework towards Healthcare

Understand

In healthcare, multimodal data is prevalent and requires to be comprehensively analyzed before diagnostic decisions, including medical images, clinical reports, etc.

• However, current large-scale artificial intelligence models predominantly focus on single-modal cognitive abilities and neglect the integration of multiple modalities.
• Therefore, we propose Stone Needle, a general multimodal large-scale model framework tailored explicitly for healthcare applications.
• Stone Needle serves as a comprehensive medical multimodal model foundation, integrating various modalities such as text, images, videos, and audio to surpass the limitations of single-modal systems.