PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection

Understand

Visual instruction tuning adapts pre-trained Multimodal Large Language Models (MLLMs) to follow human instructions for real-world applications.

• However, the rapid growth of these datasets introduces significant redundancy, leading to increased computational costs.
• Existing methods for selecting instruction data aim to prune this redundancy, but predominantly rely on computationally demanding techniques such as proxy-based inference or training-based metrics.
• Consequently, the substantial computational costs incurred by these selection processes often exacerbate the very efficiency bottlenecks they are intended to resolve, posing a significant challenge to the scalable and effective tuning of MLLMs.