Knowledge editing aims to modify outdated knowledge in language models efficiently while retaining their original capabilities. Mainstream datasets for knowledge editing are predominantly static and fail to keep in pace with the evolving real-world knowledge. In this work, we introduce CRAFT, an ever-evolving real-world dataset for knowledge editing. It evaluates models on temporal locality, common-sense locality, composite portability and alias portability, providing a comprehensive and challenging evaluation for knowledge editing, on which previous methods hardly achieve balanced performance. Towards flexible real-time knowledge editing, we propose KEDAS, a novel paradigm of knowledge editing alignment featuring diverse edit augmentation and self-adaptive post-alignment inference, exhibiting significant performance gain on both CRAFT and traditional datasets compared to previous methods. We hope this work may serve as a catalyst for shifting the focus of knowledge editing from static update to dynamic evolution.

翻译：知识编辑旨在高效修改语言模型中的过时知识，同时保持其原有能力。主流的知识编辑数据集大多为静态数据，无法跟上现实世界知识的持续演变。本文提出了CRAFT——一个持续演进的真实世界知识编辑数据集。该数据集从时间局部性、常识局部性、复合迁移性和别名迁移性四个维度评估模型，为知识编辑提供了全面且具有挑战性的评测标准，而现有方法难以在此数据集上取得均衡表现。为实现灵活的实时知识编辑，我们提出了KEDAS——一种新型的知识编辑对齐范式，其核心包括多样化编辑增强与自适应后对齐推理机制。在CRAFT及传统数据集上的实验结果表明，该方法相较于现有方法具有显著性能提升。希望本研究能推动知识编辑的研究重心从静态更新转向动态演化。