Knowledge Editing (KE) for modifying factual knowledge in Large Language Models (LLMs) has been receiving increasing attention. However, existing knowledge editing methods are entity-centric, and it is unclear whether this approach is suitable for a relation-centric perspective. To address this gap, this paper constructs a new benchmark named RaKE, which focuses on Relation based Knowledge Editing. In this paper, we establish a suite of innovative metrics for evaluation and conduct comprehensive experiments involving various knowledge editing baselines. We notice that existing knowledge editing methods exhibit the potential difficulty in their ability to edit relations. Therefore, we further explore the role of relations in factual triplets within the transformer. Our research results confirm that knowledge related to relations is not only stored in the FFN network but also in the attention layers. This provides experimental support for future relation-based knowledge editing methods.

翻译：知识编辑（Knowledge Editing，KE）旨在修改大型语言模型（Large Language Models，LLMs）中的事实知识，正日益受到关注。然而，现有知识编辑方法以实体为中心，尚不清楚该范式是否适用于以关系为中心的视角。为填补这一空白，本文构建了一个名为RaKE的新型基准，专注于基于关系的知识编辑。本文建立了一套创新的评估指标，并进行了包含多种知识编辑基线的综合实验。我们注意到，现有知识编辑方法在编辑关系的能力上存在潜在困难。因此，我们进一步探究了Transformer中事实三元组内关系的作用。研究结果证实，与关系相关的知识不仅存储在前馈神经网络（FFN）中，也存储在注意力层中，这为未来基于关系的知识编辑方法提供了实验支持。