Although existing model editing methods perform well in recalling exact edit facts, they often struggle in complex scenarios that require deeper semantic understanding rather than mere knowledge regurgitation. Leveraging the strong contextual reasoning abilities of large language models (LLMs), in-context learning (ICL) becomes a promising editing method by comprehending edit information through context encoding. However, this method is constrained by the limited context window of LLMs, leading to degraded performance and efficiency as the number of edits increases. To overcome this limitation, we propose InComeS, a flexible framework that enhances LLMs' ability to process editing contexts through explicit compression and selection mechanisms. Specifically, InComeS compresses each editing context into the key-value (KV) cache of a special gist token, enabling efficient handling of multiple edits without being restricted by the model's context window. Furthermore, specialized cross-attention modules are added to dynamically select the most relevant information from the gist pools, enabling adaptive and effective utilization of edit information. We conduct experiments on diverse model editing benchmarks with various editing formats, and the results demonstrate the effectiveness and efficiency of our method.

翻译：尽管现有模型编辑方法在精确回忆编辑事实方面表现良好，但在需要深层语义理解而非单纯知识复现的复杂场景中往往存在局限。借助大语言模型强大的上下文推理能力，上下文学习通过语境编码理解编辑信息，成为一种前景广阔的编辑方法。然而，该方法受限于LLMs有限的上下文窗口，随着编辑数量的增加，其性能与效率会显著下降。为突破这一限制，我们提出InComeS框架，该框架通过显式的压缩与选择机制增强LLMs处理编辑上下文的能力。具体而言，InComeS将每个编辑上下文压缩为特殊摘要令牌的键值缓存，从而在不受模型上下文窗口限制的前提下高效处理多重编辑。此外，框架引入专用的交叉注意力模块，动态从摘要池中选取最相关信息，实现编辑信息的自适应高效利用。我们在多种编辑格式的模型编辑基准测试中进行了实验，结果验证了该方法的有效性与高效性。