Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration

Understand

The quadratic complexity of Multimodal Large Language Models (MLLMs) with respect to context length poses significant computational and memory challenges, hindering their real-world deployment.

• In the paper, we devise a ''filter-correlate-compress'' framework to accelerate the MLLM by systematically optimizing multimodal context length during prefilling.
• The framework first implements FiCoCo-V, a training-free method operating within the vision encoder.
• It employs a redundancy-based token discard mechanism that uses a novel integrated metric to accurately filter out redundant visual tokens.