MedualTime: A Dual-Adapter Language Model for Medical Time Series-Text Multimodal Learning

Understand

The recent rapid advancements in language models (LMs) have garnered attention in medical time series-text multimodal learning.

• However, existing contrastive learning-based and prompt-based LM approaches tend to be biased, often assigning a primary role to time series modality while treating text modality as secondary.
• We classify these approaches under a temporal-primary paradigm, which may overlook the unique and critical task-relevant information embedded in text modality like clinical reports, thus failing to fully leverage mutual benefits and complementarity of different modalities.
• To fill this gap, we propose a novel textual-temporal multimodal learning paradigm that enables either modality to serve as the primary while being enhanced by the other, thereby effectively capturing modality-specific information and fostering cross-modal interaction.