MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models

Understand

In recent years, multimodal large language models (MLLMs) have significantly advanced, integrating more modalities into diverse applications.

• However, the lack of explainability remains a major barrier to their use in scenarios requiring decision transparency.
• Current neuron-level explanation paradigms mainly focus on knowledge localization or language- and domain-specific analyses, leaving the exploration of multimodality largely unaddressed.
• To tackle these challenges, we propose MINER, a transferable framework for mining modality-specific neurons (MSNs) in MLLMs, which comprises four stages: (1) modality separation, (2) importance score calculation, (3) importance score aggregation, (4) modality-specific neuron selection.