EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision

Understand

Deep generative models have enabled the automated synthesis of high-quality data for diverse applications.

• However, the most effective generative models are specialized to data from a single domain (e.g., images or text).
• Real-world applications such as healthcare require multi-modal data from multiple domains (e.g., both images and corresponding text), which are difficult to acquire due to limited availability and privacy concerns and are much harder to synthesize.
• To tackle this joint synthesis challenge, we propose an End-to-end MultImodal X-ray genERative model (EMIXER) for jointly synthesizing x-ray images and corresponding free-text reports, all conditional on diagnosis labels.