Model editing has recently gained widespread attention. Current model editing methods primarily involve modifying model parameters or adding additional modules to the existing model. However, the former causes irreversible damage to Large Language Models (LLMs), while the latter incurs additional inference overhead and fuzzy vector matching is not always reliable. To address these issues, we propose an expandable Subject Word Embedding Altering (SWEA) framework, which finds the fused embeddings through character-level key-value matching and adds them to the subject word embeddings in Transformer input. To get these fused embeddings, we propose optimizing then suppressing fusion method, which first optimizes learnable embedding vectors for the editing target and then suppresses the Knowledge Embedding Dimensions (KEDs) to obtain final fused embeddings. We thus propose SWEA$\oplus$OS method for editing factual knowledge in LLMs. We demonstrate the overall state-of-the-art (SOTA) performance of SWEA$\oplus$OS on the COUNTERFACT and zsRE datasets. To further validate the reasoning ability of SWEA$\oplus$OS in editing knowledge, we evaluate it on the more complex RippleEdits benchmark. The results demonstrate that SWEA$\oplus$OS possesses SOTA reasoning ability.

翻译：模型编辑近年来受到广泛关注。当前模型编辑方法主要包括修改模型参数或为现有模型添加附加模块。然而，前者会对大型语言模型（LLMs）造成不可逆的损伤，而后者则引入额外的推理开销，且模糊向量匹配并非总是可靠。为解决这些问题，我们提出一种可扩展的主语词嵌入调整（SWEA）框架，该框架通过字符级键值匹配找到融合嵌入，并将其添加到Transformer输入中的主语词嵌入中。为获得这些融合嵌入，我们提出一种先优化后抑制的融合方法，即首先针对编辑目标优化可学习的嵌入向量，然后抑制知识嵌入维度（KEDs），从而得到最终的融合嵌入。据此，我们提出了SWEA$\oplus$OS方法，用于编辑LLMs中的事实知识。我们在COUNTERFACT和zsRE数据集上展示了SWEA$\oplus$OS的整体最优（SOTA）性能。为进一步验证SWEA$\oplus$OS在知识编辑中的推理能力，我们在更复杂的RippleEdits基准上对其进行了评估。结果表明，SWEA$\oplus$OS具备SOTA推理能力。