Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. However, a primary limitation lies in the significant computational demands during training, arising from their extensive parameterization. This challenge is further intensified by the dynamic nature of the world, necessitating frequent updates to LLMs to correct outdated information or integrate new knowledge, thereby ensuring their continued relevance. Note that many applications demand continual model adjustments post-training to address deficiencies or undesirable behaviors. There is an increasing interest in efficient, lightweight methods for on-the-fly model modifications. To this end, recent years have seen a burgeoning in the techniques of knowledge editing for LLMs, which aim to efficiently modify LLMs' behaviors within specific domains while preserving overall performance across various inputs. In this paper, we first define the knowledge editing problem and then provide a comprehensive review of cutting-edge approaches. Drawing inspiration from educational and cognitive research theories, we propose a unified categorization criterion that classifies knowledge editing methods into three groups: resorting to external knowledge, merging knowledge into the model, and editing intrinsic knowledge. Furthermore, we introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches. Additionally, we provide an in-depth analysis of knowledge location, which can provide a deeper understanding of the knowledge structures inherent within LLMs. Finally, we discuss several potential applications of knowledge editing, outlining its broad and impactful implications.

翻译：大型语言模型在理解和生成与人类沟通高度相似的文本方面展现出非凡能力。然而，其主要局限在于训练过程中因广泛参数化而产生的巨大计算需求。这一挑战因世界的动态性质而进一步加剧，需要频繁更新大型语言模型以纠正过时信息或整合新知识，从而确保其持续相关性。值得注意的是，许多应用要求模型在训练后进行持续调整以解决缺陷或不良行为。人们对高效、轻量级的实时模型修改方法越来越感兴趣。为此，近年来针对大型语言模型的知识编辑技术蓬勃发展，这些技术旨在高效修改模型在特定领域内的行为，同时保持其在各种输入上的整体性能。在本文中，我们首先定义知识编辑问题，然后全面综述前沿方法。借鉴教育和认知研究理论，我们提出统一的分类标准，将知识编辑方法分为三类：借助外部知识、将知识融入模型以及编辑固有知识。此外，我们引入新基准KnowEdit，用于对代表性知识编辑方法进行全面实证评估。同时，我们对知识定位进行深入分析，以加深对大型语言模型中固有知识结构的理解。最后，我们讨论知识编辑的若干潜在应用，概述其广泛而深远的影响。