Knowledge editing aims to modify outdated knowledge in language models efficiently while retaining their original capabilities. Mainstream datasets for knowledge editing are predominantly static and fail to keep in pace with the evolving real-world knowledge. In this work, we introduce CRAFT, an ever-evolving real-world dataset for knowledge editing. It evaluates models on temporal locality, common-sense locality, composite portability and alias portability, providing a comprehensive and challenging evaluation for knowledge editing, on which previous methods hardly achieve balanced performance. Towards flexible real-time knowledge editing, we propose KEDAS, a novel paradigm of knowledge editing alignment featuring diverse edit augmentation and self-adaptive post-alignment inference, exhibiting significant performance gain on both CRAFT and traditional datasets compared to previous methods. We hope this work may serve as a catalyst for shifting the focus of knowledge editing from static update to dynamic evolution.

翻译：知识编辑旨在高效修改语言模型中的过时知识，同时保留其原有能力。主流的知识编辑数据集以静态为主，无法跟上现实世界知识的持续演进。在本工作中，我们引入了CRAFT——一个持续演进的现实世界知识编辑数据集。该数据集从时间局部性、常识局部性、复合可迁移性和别名可迁移性四个维度评估模型，为知识编辑提供了全面且具有挑战性的评估标准，而此前的方法难以在此类评估中取得均衡表现。为实现灵活的实时知识编辑，我们提出了KEDAS——一种新颖的知识编辑对齐范式，其核心包括多样化编辑增强与自适应后对齐推理，在CRAFT及传统数据集上相较于此前方法均展现出显著的性能提升。我们希望这项工作能推动知识编辑的研究重心从静态更新转向动态演化。