Weight-preserving model editing techniques heavily rely on the scoping mechanism that decides when to apply an edit to the base model. These scoping mechanisms utilize distance functions in the representation space to ascertain the scope of the edit. In this work, we show that distance-based scoping functions grapple with lexical biases leading to issues such as misfires with irrelevant prompts that share similar lexical characteristics. To address this problem, we introduce, Projector Editor Networks for Model Editing (PENME),is a model editing approach that employs a compact adapter with a projection network trained via a contrastive learning objective. We demonstrate the efficacy of PENME in achieving superior results while being compute efficient and flexible to adapt across model architectures.

翻译：权重保持型模型编辑技术高度依赖于范围界定机制，该机制决定何时对基础模型应用编辑。这些范围界定机制利用表示空间中的距离函数来确定编辑的适用范围。本研究发现，基于距离的范围界定函数存在词汇偏差问题，导致在与编辑内容无关但具有相似词汇特征的提示上产生误触发。为解决此问题，我们提出投影器编辑网络模型编辑方法（PENME），该方法采用配备投影网络的紧凑适配器，通过对比学习目标进行训练。实验证明PENME在实现优异性能的同时，兼具计算高效性和跨模型架构的适应灵活性。