Recently, knowledge editing (KE) has emerged as a promising approach to update specific facts in Large Language Models (LLMs) without the need for full retraining. Despite the effectiveness in general-domain benchmarks, their applicability to complex medical domain remains largely unexplored. Medical knowledge editing is particularly challenging, as it requires LLMs to internalize the knowledge and generalize to unseen scenarios for effective and interpretable decision-making. In this work, we propose a novel framework called MedEditBench to rigorously evaluate the effectiveness of existing KE methods in the medical domain. In MedEditBench, we introduce a new medical knowledge editing benchmark as well as three different knowledge editing paradigms, which are designed to assess the impact of different knowledge sources for editing. Our findings indicate that current KE methods result in only superficial memorization of the injected information, failing to generalize to new scenarios. To overcome this limitation, we present Self-Generated Rationale Editing (SGR-Edit), which utilizes model-derived rationales as the target knowledge for editing, thereby uncovering the underlying reasoning process and demonstrating significant improvements over existing KE approaches. Additionally, we offer deeper insights into medical knowledge editing, including the localization of medical knowledge in LLMs and the impact of sequential editing on evolving knowledge. This could provide practical guidance for implementing KE methods in real-world medical applications.

翻译：近年来，知识编辑（KE）已成为一种有前景的方法，用于更新大型语言模型（LLM）中的特定事实，而无需进行完整的重新训练。尽管在通用领域基准测试中显示出有效性，但其在复杂医学领域的适用性在很大程度上仍未得到探索。医学知识编辑尤其具有挑战性，因为它要求LLM内化知识并将其推广到未见过的场景，以实现有效且可解释的决策。在本工作中，我们提出了一个名为MedEditBench的新颖框架，用于严谨评估现有KE方法在医学领域的有效性。在MedEditBench中，我们引入了一个新的医学知识编辑基准以及三种不同的知识编辑范式，这些范式旨在评估不同知识来源对编辑的影响。我们的研究结果表明，当前的KE方法仅导致对注入信息的表面记忆，无法推广到新场景。为了克服这一限制，我们提出了自生成推理编辑（SGR-Edit），该方法利用模型衍生的推理作为编辑的目标知识，从而揭示底层的推理过程，并展示出相对于现有KE方法的显著改进。此外，我们提供了对医学知识编辑更深入的见解，包括医学知识在LLM中的定位以及顺序编辑对演化知识的影响。这可为在实际医学应用中实施KE方法提供实用指导。