Knowledge Editing (KE) has emerged as a promising paradigm for updating facts in Large Language Models (LLMs) without retraining. However, progress in Multilingual Knowledge Editing (MKE) is currently hindered by biased evaluation frameworks. We observe that existing MKE benchmarks are typically constructed by mechanically translating English-centric datasets into target languages (e.g., English-to-Chinese). This approach introduces translation artifacts and neglects culturally specific entities native to the target language, failing to reflect the true knowledge distribution of LLMs. To address this, we propose CLM-Bench, a culture-aware benchmark constructed using a native Chinese-first methodology. We curate 1,010 high-quality CounterFact pairs rooted in Chinese cultural contexts and align them with English counterparts. Using CLM-Bench, we conduct extensive experiments on representative LLMs (e.g., Llama-3, Qwen2) and reveal a significant Cross-lingual Misalignment: edits in one language function independently and fail to propagate to the other. We further provide a geometric explanation via layer-wise representation analysis, demonstrating that edit vectors for Chinese and English are nearly orthogonal -- residing in disjoint subspaces -- while mixed-lingual editing exhibits linear additivity of these vectors. Our findings challenge the effectiveness of current methods in cross-lingual transfer and underscore the importance of culturally native benchmarks.
翻译:知识编辑已成为一种无需重新训练即可更新大语言模型中事实的有前景范式。然而,当前多语言知识编辑的发展受到评估框架偏见的阻碍。我们观察到,现有的多语言知识编辑基准通常通过机械地将以英语为中心的数据集翻译成目标语言(例如英译中)来构建。这种方法引入了翻译伪影,并忽略了目标语言中固有的文化特定实体,未能反映大语言模型的真实知识分布。为解决此问题,我们提出了CLM-Bench,一个采用原生中文优先方法构建的文化感知基准。我们整理了1,010对植根于中文文化语境的高质量CounterFact配对,并将其与英文对应项对齐。利用CLM-Bench,我们对代表性大语言模型(如Llama-3、Qwen2)进行了广泛实验,揭示了一个显著的跨语言错位现象:一种语言中的编辑独立运作,无法传播到另一种语言。我们进一步通过分层表示分析提供了几何解释,证明中文和英文的编辑向量近乎正交——存在于不相交的子空间中——而混合语言编辑则表现出这些向量的线性可加性。我们的研究结果挑战了当前方法在跨语言迁移中的有效性,并强调了文化原生基准的重要性。