Recently, there has been a growing interest in knowledge editing for Large Language Models (LLMs). Current approaches and evaluations merely explore the instance-level editing, while whether LLMs possess the capability to modify concepts remains unclear. This paper pioneers the investigation of editing conceptual knowledge for LLMs, by constructing a novel benchmark dataset ConceptEdit and establishing a suite of new metrics for evaluation. The experimental results reveal that, although existing editing methods can efficiently modify concept-level definition to some extent, they also have the potential to distort the related instantial knowledge in LLMs, leading to poor performance. We anticipate this can inspire further progress in better understanding LLMs. Our project homepage is available at https://zjunlp.github.io/project/ConceptEdit.

翻译：近期，针对大型语言模型的知识编辑研究日益兴起。当前方法与评估仅涉及实例层面的编辑，而大型语言模型是否具备修改概念的能力尚不明确。本文通过构建新型基准数据集ConceptEdit并建立一套全新评估指标，率先探索了大型语言模型的概念知识编辑。实验结果表明，现有编辑方法虽能在一定程度上有效修改概念级定义，但同时也可能扭曲大模型中相关的实例性知识，导致性能不佳。我们期望此项工作能激发对大型语言模型更深层次理解的进一步研究。项目主页详见https://zjunlp.github.io/project/ConceptEdit。