Large language models (LLMs) increasingly rely on knowledge editing to support knowledge-intensive reasoning, but this flexibility also introduces critical safety risks: adversaries can inject malicious or misleading knowledge that corrupts downstream reasoning and leads to harmful outcomes. Existing knowledge editing benchmarks primarily focus on editing efficacy and lack a unified framework for systematically evaluating the safety implications of edited knowledge on reasoning behavior. To address this gap, we present EditRisk-Bench, a benchmark for systematically evaluating safety risks of knowledge-intensive reasoning under malicious knowledge editing. Unlike prior benchmarks that mainly emphasize edit success, generalization, and locality, EditRisk-Bench focuses on how injected knowledge affects downstream reasoning behavior and reliability. It integrates diverse malicious scenarios, including misinformation, bias, and safety violations, together with multi-level knowledge-intensive reasoning tasks and representative editing strategies within a unified evaluation framework measuring attack effectiveness, reasoning correctness, and side effects. Extensive experiments on both open-source and closed-source LLMs show that malicious knowledge editing can reliably induce incorrect or unsafe reasoning while largely preserving general capabilities, making such risks difficult to detect. We further identify several key factors influencing these risks, including edit scale, knowledge characteristics, and reasoning complexity. EditRisk-Bench provides an extensible testbed for understanding and mitigating safety risks in knowledge editing for LLMs.
翻译:大语言模型(LLMs)日益依赖知识编辑来支持知识密集型推理,但这种灵活性也带来了关键安全风险:攻击者可注入恶意或误导性知识,破坏下游推理并导致有害结果。现有知识编辑基准主要聚焦于编辑效能,缺乏系统评估编辑知识对推理行为安全影响的统一框架。为弥补这一空白,我们提出EditRisk-Bench——一个系统评估恶意知识编辑下知识密集型推理安全风险的基准。不同于以往侧重编辑成功性、泛化性与局部性的基准,EditRisk-Bench聚焦于注入知识如何影响下游推理行为与可靠性。它整合了包含错误信息、偏见和安全违规在内的多样化恶意场景,结合多层次知识密集型推理任务与代表性编辑策略,在统一评估框架中衡量攻击有效性、推理正确性与副作用。对开源和闭源LLMs的大量实验表明,恶意知识编辑可可靠地诱发错误或非安全推理,同时大体保持通用能力,使得此类风险难以检测。我们进一步识别出影响这些风险的关键因素,包括编辑规模、知识特性与推理复杂度。EditRisk-Bench为理解和缓解LLMs知识编辑中的安全风险提供了可扩展的测试平台。