Model editing is a growing area focused on updating the knowledge embedded within models. Among the various methodologies, ROME and MEMIT stand out as leading "locate-and-edit" model editing techniques. While MEMIT enables batched editing of memories, ROME is limited to changing one fact at a time. This paper introduces a unifying framework that brings ROME and MEMIT under a single conceptual umbrella, optimizing for the same goal, which we call the "preservation-memorization" objective. This objective aims to preserve the representations of certain selected vectors while memorizing the representations of new factual information. Specifically, ROME optimizes this objective using an equality constraint, whereas MEMIT employs a more flexible least-square constraint. In addition to making batched edits, MEMIT also edits the model at multiple layers. We disentangle the distribution of edits to multiple layers from the optimization objective of MEMIT and show that these edit-distribution algorithms should be considered separate entities worthy of their own line of research. Finally, we present EMMET - an Equality-constrained Mass Model Editing algorithm for Transformers, a new batched memory-editing algorithm. With EMMET, we present a closed form solution for the equality-constrained version of the preservation-memorization objective. We show that EMMET is able to perform batched-edits on par with MEMIT up to a batch-size of 256 and discuss the challenges in stabilizing EMMET. By articulating the "locate-and-edit" model editing algorithms under a simple conceptual framework of "preservation-memorization", we aim to bridge the gap between intuition and mathematics and hope to simplify the journey for future researchers in model editing.

翻译：模型编辑是一个日益发展的领域，专注于更新模型中嵌入的知识。在众多方法中，ROME和MEMIT是领先的“定位与编辑”模型编辑技术。MEMIT能够批量编辑记忆，而ROME每次仅能修改一个事实。本文提出了一个统一框架，将ROME和MEMIT纳入同一个概念体系，并优化同一目标，我们称之为“保留-记忆”目标。该目标旨在保留特定选定向量的表征，同时记忆新事实信息的表征。具体而言，ROME通过等式约束优化这一目标，而MEMIT则采用更灵活的最小二乘约束。除实现批量编辑外，MEMIT还在模型的多个层进行编辑。我们将编辑在多层的分布从MEMIT的优化目标中解耦，表明这些编辑分布算法应被视为独立的研究方向。最后，我们提出了EMMET——一种基于等式约束的Transformer批量模型编辑算法。借助EMMET，我们给出了“保留-记忆”目标在等式约束条件下的闭式解。实验表明，在批量大小不超过256的情况下，EMMET的批量编辑性能与MEMIT相当，同时我们讨论了稳定EMMET所面临的挑战。通过将“定位与编辑”模型编辑算法置于“保留-记忆”这一简洁的概念框架下，我们旨在弥合直觉与数学之间的差距，并希望简化未来模型编辑研究者的探索之路。