Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial. Existing communication protocols either rely on natural language, incurring high inference costs and information loss, or on hidden states, which suffer from information concentration bias and inefficiency. To address these limitations, we propose KVComm, a novel communication framework that enables efficient communication between LLMs through selective sharing of KV pairs. KVComm leverages the rich information encoded in the KV pairs while avoiding the pitfalls of hidden states. We introduce a KV layer-wise selection strategy based on attention importance scores with a Gaussian prior to identify the most informative KV pairs for communication. Extensive experiments across diverse tasks and model pairs demonstrate that KVComm achieves comparable performance to the upper-bound method, which directly merges inputs to one model without any communication, while transmitting as few as 30\% of layers' KV pairs. Our study highlights the potential of KV pairs as an effective medium for inter-LLM communication, paving the way for scalable and efficient multi-agent systems.

翻译：大语言模型（LLMs）正日益部署于多智能体系统中，其中有效的模型间通信至关重要。现有的通信协议要么依赖自然语言（导致高昂的推理成本和信息损失），要么依赖隐藏状态（存在信息集中偏差和效率低下的问题）。为应对这些局限性，我们提出了KVComm，一种新颖的通信框架，通过选择性共享KV对来实现LLMs之间的高效通信。KVComm利用了KV对中编码的丰富信息，同时避免了隐藏状态的缺陷。我们引入了一种基于注意力重要性分数并带有高斯先验的KV分层选择策略，以识别出最具信息量的KV对用于通信。在多样化任务和模型对上的大量实验表明，KVComm实现了与上界方法（即直接将输入合并到一个模型而无需任何通信）相当的性能，同时仅传输低至30%层数的KV对。我们的研究凸显了KV对作为LLM间通信有效媒介的潜力，为可扩展且高效的多智能体系统铺平了道路。