Recently, while large language models (LLMs) have demonstrated impressive results, they still suffer from hallucination, i.e., the generation of false information. Model editing is the task of fixing factual mistakes in LLMs; yet, most previous works treat it as a one-time task, paying little attention to ever-emerging mistakes generated by LLMs. We address the task of sequential model editing (SME) that aims to rectify mistakes continuously. A Dynamic Auxiliary Fusion Network (DAFNet) is designed to enhance the semantic interaction among the factual knowledge within the entire sequence, preventing catastrophic forgetting during the editing process of multiple knowledge triples. Specifically, (1) for semantic fusion within a relation triple, we aggregate the intra-editing attention flow into auto-regressive self-attention with token-level granularity in LLMs. We further leverage multi-layer diagonal inter-editing attention flow to update the weighted representations of the entire sequence-level granularity. (2) Considering that auxiliary parameters are required to store the knowledge for sequential editing, we construct a new dataset named \textbf{DAFSet}, fulfilling recent, popular, long-tail and robust properties to enhance the generality of sequential editing. Experiments show DAFNet significantly outperforms strong baselines in single-turn and sequential editing. The usage of DAFSet also consistently improves the performance of other auxiliary network-based methods in various scenarios

翻译：近年来，尽管大语言模型（LLMs）已展现出令人瞩目的成果，但其仍存在幻觉问题，即生成虚假信息。模型编辑旨在修正大语言模型中的事实性错误；然而，现有研究多将其视为一次性任务，未能充分关注大语言模型持续产生的新错误。本文致力于序列化模型编辑（SME）任务，旨在持续修正模型错误。我们设计了动态辅助融合网络（DAFNet），通过增强整个序列中事实知识间的语义交互，防止在编辑多个知识三元组过程中发生灾难性遗忘。具体而言：（1）针对关系三元组内部的语义融合，我们将编辑内部注意力流以词元级粒度聚合至大语言模型的自回归自注意力机制中。进一步利用多层对角跨编辑注意力流，以序列级粒度更新加权表示。（2）考虑到序列化编辑需要辅助参数存储知识，我们构建了名为 \textbf{DAFSet} 的新数据集，该数据集具备时效性、流行性、长尾性和鲁棒性特征，以增强序列化编辑的泛化能力。实验表明，DAFNet 在单轮编辑和序列化编辑中均显著优于现有基线方法。DAFSet 的使用也持续提升了其他基于辅助网络方法在多种场景下的性能表现。