Medical Vision Generalist: Unifying Medical Imaging Tasks in Context

Understand

This study presents Medical Vision Generalist (MVG), the first foundation model capable of handling various medical imaging tasks -- such as cross-modal synthesis, image segmentation, denoising, and inpainting -- within a unified image-to-image generation framework.

• Specifically, MVG employs an in-context generation strategy that standardizes the handling of inputs and outputs as images.
• By treating these tasks as an image generation process conditioned on prompt image-label pairs and input images, this approach enables a flexible unification of various tasks, even those spanning different modalities and datasets.
• To capitalize on both local and global context, we design a hybrid method combining masked image modeling with autoregressive training for conditional image generation.