KVComm: Enabling Efficient LLM Communication through Selective KV Sharing

Understand

Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial.

• Existing communication protocols either rely on natural language, incurring high inference costs and information loss, or on hidden states, which suffer from information concentration bias and inefficiency.
• To address these limitations, we propose KVComm, a novel communication framework that enables efficient communication between LLMs through selective sharing of KV pairs.
• KVComm leverages the rich information encoded in the KV pairs while avoiding the pitfalls of hidden states.