Model editing is an emerging field that focuses on updating the knowledge embedded within large language models (LLMs) without extensive retraining. However, current model editing methods significantly compromise the general abilities of LLMs as the number of edits increases, and this trade-off poses a substantial challenge to the continual learning of LLMs. In this paper, we first theoretically analyze that the factor affecting the general abilities in sequential model editing lies in the condition number of the edited matrix. The condition number of a matrix represents its numerical sensitivity, and therefore can be used to indicate the extent to which the original knowledge associations stored in LLMs are perturbed after editing. Subsequently, statistical findings demonstrate that the value of this factor becomes larger as the number of edits increases, thereby exacerbating the deterioration of general abilities. To this end, a framework termed Perturbation Restraint on Upper bouNd for Editing (PRUNE) is proposed, which applies the condition number restraints in sequential editing. These restraints can lower the upper bound on perturbation to edited models, thus preserving the general abilities. Systematically, we conduct experiments employing three popular editing methods on three LLMs across four representative downstream tasks. Evaluation results show that PRUNE can preserve considerable general abilities while maintaining the editing performance effectively in sequential model editing. The code and data are available at https://github.com/mjy1111/PRUNE.

翻译：模型编辑是一个新兴领域，专注于在不进行大规模重新训练的情况下更新大型语言模型（LLMs）中嵌入的知识。然而，随着编辑次数的增加，当前的模型编辑方法会显著损害LLMs的通用能力，这种权衡对LLMs的持续学习构成了重大挑战。在本文中，我们首先从理论上分析了影响序列模型编辑中通用能力的因素在于被编辑矩阵的条件数。矩阵的条件数代表了其数值敏感性，因此可用于指示编辑后LLMs中存储的原始知识关联被扰动的程度。随后的统计发现表明，该因子的值随着编辑次数的增加而变大，从而加剧了通用能力的退化。为此，我们提出了一个名为“编辑上界扰动抑制”（PRUNE）的框架，该框架在序列编辑中应用条件数约束。这些约束可以降低对被编辑模型扰动的上界，从而保留通用能力。系统性地，我们在三个LLMs上采用三种流行的编辑方法，在四个代表性的下游任务中进行了实验。评估结果表明，PRUNE能够在序列模型编辑中有效保持编辑性能的同时，显著保留通用能力。代码和数据可在 https://github.com/mjy1111/PRUNE 获取。