ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding

Understand

Automatic chart understanding is crucial for content comprehension and document parsing.

• Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in chart understanding through domain-specific alignment and fine-tuning.
• However, current MLLMs still struggle to provide faithful data and reliable analysis only based on charts.
• To address it, we propose ChartMoE, which employs the Mixture of Expert (MoE) architecture to replace the traditional linear projector to bridge the modality gap.