MPIC: Position-Independent Multimodal Context Caching System for Efficient MLLM Serving

Understand

The context caching technique is employed to accelerate the Multimodal Large Language Model (MLLM) inference by prevailing serving platforms currently.

• However, this approach merely reuses the Key-Value (KV) cache of the initial sequence of prompt, resulting in full KV cache recomputation even if the prefix differs slightly.
• This becomes particularly inefficient in the context of interleaved text and images, as well as multimodal retrieval-augmented generation.
• This paper proposes position-independent caching as a more effective approach for multimodal information management.