Efficient knowledge editing of large language models is crucial for replacing obsolete information or incorporating specialized knowledge on a large scale. However, previous methods implicitly assume that knowledge is localized and isolated within the model, an assumption that oversimplifies the interconnected nature of model knowledge. The premise of localization results in an incomplete knowledge editing, whereas an isolated assumption may impair both other knowledge and general abilities. It introduces instability to the performance of the knowledge editing method. To transcend these assumptions, we introduce StableKE, a method adopts a novel perspective based on knowledge augmentation rather than knowledge localization. To overcome the expense of human labeling, StableKE integrates two automated knowledge augmentation strategies: Semantic Paraphrase Enhancement strategy, which diversifies knowledge descriptions to facilitate the teaching of new information to the model, and Contextual Description Enrichment strategy, expanding the surrounding knowledge to prevent the forgetting of related information. StableKE surpasses other knowledge editing methods, demonstrating stability both edited knowledge and multi-hop knowledge, while also preserving unrelated knowledge and general abilities. Moreover, StableKE can edit knowledge on ChatGPT.
翻译:高效的知识编辑对于大规模替换过时信息或整合专业知识至关重要。然而,现有方法隐含地假设知识在模型中是局域化和孤立的,这一假设过度简化了模型知识间的相互关联特性。局域化前提导致知识编辑不完整,而孤立性假设可能损害其他知识及通用能力,从而引发知识编辑方法性能的不稳定性。为突破这些假设,我们提出StableKE方法,其基于知识增强而非知识局域化的创新视角。为克服人工标注的高成本,StableKE整合两种自动化知识增强策略:语义释义增强策略通过多样化知识描述促进模型学习新信息,上下文描述丰富策略则扩展周边知识以防止相关信息的遗忘。StableKE在编辑知识及多跳知识上均展现稳定性,同时保留无关知识与通用能力,性能超越现有知识编辑方法。此外,StableKE还能在ChatGPT上实现知识编辑。