Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. However, a primary limitation lies in the significant computational demands during training, arising from their extensive parameterization. This challenge is further intensified by the dynamic nature of the world, necessitating frequent updates to LLMs to correct outdated information or integrate new knowledge, thereby ensuring their continued relevance. Note that many applications demand continual model adjustments post-training to address deficiencies or undesirable behaviors. There is an increasing interest in efficient, lightweight methods for on-the-fly model modifications. To this end, recent years have seen a burgeoning in the techniques of knowledge editing for LLMs, which aim to efficiently modify LLMs' behaviors within specific domains while preserving overall performance across various inputs. In this paper, we first define the knowledge editing problem and then provide a comprehensive review of cutting-edge approaches. Drawing inspiration from educational and cognitive research theories, we propose a unified categorization criterion that classifies knowledge editing methods into three groups: resorting to external knowledge, merging knowledge into the model, and editing intrinsic knowledge. Furthermore, we introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches. Additionally, we provide an in-depth analysis of knowledge location, which can provide a deeper understanding of the knowledge structures inherent within LLMs. Finally, we discuss several potential applications of knowledge editing, outlining its broad and impactful implications.

翻译：大语言模型（LLMs）在理解和生成接近人类交流的文本方面展现出非凡能力。然而，其主要局限性在于训练过程中因参数规模庞大而产生的巨大计算需求。这一挑战因世界的动态性而进一步加剧，需要频繁更新LLMs以修正过时信息或整合新知识，从而确保其持续相关性。值得注意的是，许多应用要求模型在训练后持续调整以解决缺陷或不良行为。当前，对高效、轻量级的实时模型修改方法兴趣日益增长。为此，近年来面向LLMs的知识编辑技术蓬勃发展，旨在高效修改LLMs在特定领域内的行为，同时保持其在各种输入上的整体性能。本文首先定义了知识编辑问题，随后对前沿方法进行了全面综述。受教育和认知研究理论的启发，我们提出统一的分类标准，将知识编辑方法分为三类：借助外部知识、将知识融入模型以及编辑内在知识。此外，我们引入新基准KnowEdit，用于对代表性知识编辑方法进行全面的实证评估。我们还深入分析了知识定位问题，以加深对LLMs内在知识结构的理解。最后，我们讨论了知识编辑的若干潜在应用，阐述了其广泛而深远的影响。