Modern large language models (LLMs) are inherently auto-regressive, requiring input to be serialized into flat sequences regardless of their structural dependencies. This serialization hinders the model's ability to leverage structural inductive biases, especially in tasks such as retrieval-augmented generation (RAG) and reasoning on data with native graph structures, where inter-segment dependencies are crucial. We introduce Graph-KV with the potential to overcome this limitation. Graph-KV leverages the KV-cache of text segments as condensed representations and governs their interaction through structural inductive biases. In this framework, 'target' segments selectively attend only to the KV-caches of their designated 'source' segments, rather than all preceding segments in a serialized sequence. This approach induces a graph-structured block mask, sparsifying attention and enabling a message-passing-like step within the LLM. Furthermore, strategically allocated positional encodings for source and target segments reduce positional bias and context window consumption. We evaluate Graph-KV across three scenarios: (1) seven RAG benchmarks spanning direct inference, multi-hop reasoning, and long-document understanding; (2) Arxiv-QA, a novel academic paper QA task with full-text scientific papers structured as citation ego-graphs; and (3) paper topic classification within a citation network. By effectively reducing positional bias and harnessing structural inductive biases, Graph-KV substantially outperforms baselines, including standard costly sequential encoding, across various settings. Code and the Graph-KV data are publicly available.

翻译：现代大语言模型本质上是自回归的，无论输入数据是否存在结构依赖关系，都需要将其序列化为扁平序列。这种序列化限制了模型利用结构归纳偏置的能力，尤其在检索增强生成任务及原生图结构数据的推理任务中，跨片段依赖关系至关重要。我们提出的Graph-KV具备突破此限制的潜力。该方法利用文本片段的KV缓存作为压缩表示，并通过结构归纳偏置调控其交互机制。在此框架中，"目标"片段仅选择性关注其指定"源"片段的KV缓存，而非序列化序列中的所有前序片段。该方法诱导出图结构化的块掩码，实现了注意力机制的稀疏化，并在大语言模型内部启用了类消息传递的步骤。此外，为源片段和目标片段策略性分配的位置编码有效降低了位置偏置并减少了上下文窗口的消耗。我们在三种场景下评估Graph-KV：（1）涵盖直接推理、多跳推理和长文档理解的七个检索增强生成基准测试；（2）Arxiv-QA——以引文自我图结构组织全文科学论文的新型学术论文问答任务；（3）引文网络中的论文主题分类。通过有效降低位置偏置并充分利用结构归纳偏置，Graph-KV在多种设定下显著优于基线方法（包括标准的高成本序列编码方案）。相关代码与Graph-KV数据已公开。