Model editing aims to efficiently alter the behavior of Large Language Models (LLMs) within a desired scope, while ensuring no adverse impact on other inputs. Recent years have witnessed various model editing methods been proposed. However, these methods either exhibit poor overall performance or struggle to strike a balance between generalization and locality. We propose MOMoE, a model editing adapter utilizing a Mixture of Experts (MoE) architecture with a knowledge anchor routing strategy. MOMoE updates knowledge using a bypass MoE structure, keeping the original parameters unchanged to preserve the general ability of LLMs. And, the knowledge anchor routing ensures that inputs requiring similar knowledge are routed to the same expert, thereby enhancing the generalization of the updated knowledge. Experimental results show the superiority of our approach over both batch editing and sequential batch editing tasks, exhibiting exceptional overall performance alongside outstanding balance between generalization and locality. Our code will be available.

翻译：模型编辑旨在高效地改变大型语言模型（LLM）在特定范围内的行为，同时确保对其他输入不产生负面影响。近年来，各种模型编辑方法相继被提出。然而，这些方法要么整体性能较差，要么难以在泛化性与局部性之间取得平衡。我们提出MOMoE，一种采用专家混合（MoE）架构与知识锚定路由策略的模型编辑适配器。MOMoE通过旁路MoE结构更新知识，保持原始参数不变以保留LLM的通用能力。同时，知识锚定路由机制确保需要相似知识的输入被路由至同一专家，从而增强更新知识的泛化能力。实验结果表明，我们的方法在批量编辑和顺序批量编辑任务上均优于现有方法，展现出卓越的整体性能以及在泛化性与局部性之间的出色平衡。我们的代码将公开提供。