The potential of using a large language model (LLM) as a knowledge base (KB) has sparked significant interest. To manage the knowledge acquired by LLMs, we need to ensure that the editing of learned facts respects internal logical constraints, which are known as dependency of knowledge. Existing work on editing LLMs has partially addressed the issue of dependency, when the editing of a fact should apply to its lexical variations without disrupting irrelevant ones. However, they neglect the dependency between a fact and its logical implications. We propose an evaluation protocol with an accompanying question-answering dataset, DepEdit, that provides a comprehensive assessment of the editing process considering the above notions of dependency. Our protocol involves setting up a controlled environment in which we edit facts and monitor their impact on LLMs, along with their implications based on If-Then rules. Extensive experiments on DepEdit show that existing knowledge editing methods are sensitive to the surface form of knowledge, and that they have limited performance in inferring the implications of edited facts.

翻译：大型语言模型（LLM）作为知识库（KB）的潜力引发了广泛关注。为管理LLM获取的知识，需确保对已习得事实的编辑遵循内部逻辑约束，即知识的依赖关系。现有LLM编辑研究已部分解决依赖问题——事实编辑应作用于其词法变体，同时避免干扰无关内容，但忽略了事实与其逻辑隐含之间的依赖关系。我们提出评估协议及配套问答数据集DepEdit，为上述依赖概念下的编辑过程提供全面评估。该协议通过构建受控环境，在编辑事实的同时监测其对LLM的影响，并基于“如果-则”规则分析事实的隐含影响。在DepEdit上的大量实验表明，现有知识编辑方法对知识的表层形式敏感，且在推断编辑事实的隐含信息方面性能有限。