Recent advances in large language models (LLMs) have opened up new paradigms for accessing the knowledge stored in their parameters. One critical challenge that has emerged is the presence of hallucinations in LLM outputs due to false or outdated knowledge. Since retraining LLMs with updated information is resource-intensive, there has been a growing interest in model editing. However, many model editing methods, while effective in various scenarios, tend to overemphasize aspects such as efficacy, generalization, and locality in editing performance, often overlooking potential side effects on the general abilities of LLMs. In this paper, we raise concerns that the improvement of model factuality may come at the cost of a significant degradation of these general abilities, which is not conducive to the sustainable development of LLMs. Systematically, we analyze side effects by evaluating four popular editing methods on two LLMs across eight representative task categories. Extensive empirical research reveals that model editing does improve model factuality but at the expense of substantially impairing general abilities. Therefore, we advocate for more research efforts to minimize the loss of general abilities acquired during LLM pre-training and to ultimately preserve them during model editing.

翻译：近年来，大型语言模型（LLM）的进展为访问其参数中存储的知识开辟了新范式。其中一个关键挑战是由于错误或过时知识导致的LLM输出中存在幻觉现象。由于用更新信息重新训练LLM需要大量资源，模型编辑逐渐受到关注。然而，许多模型编辑方法虽然在多种场景下有效，却往往过度强调编辑性能中的有效性、泛化性和局部性，常常忽视对LLM通用能力的潜在副作用。本文提出担忧：模型事实性的提升可能以显著降低这些通用能力为代价，这不利于LLM的可持续发展。我们通过评估两种LLM上四种流行编辑方法在八个代表性任务类别中的表现，系统性分析了副作用。大量实证研究表明，模型编辑虽然提升了模型事实性，但严重损害了通用能力。因此，我们呼吁更多研究致力于最小化LLM预训练中获得的通用能力的损失，并在模型编辑过程中最终保留这些能力。