Even the most advanced language models remain susceptible to errors necessitating to modify these models without initiating a comprehensive retraining process. Model editing refers to the modification of a model's knowledge or representations in a manner that produces the desired outcomes. Prior research primarily centered around editing factual data e.g. "Messi plays for Inter Miami" confining the definition of an edit to a knowledge triplet i.e. (subject, object, relation). However, as the applications of language models expand, so do the diverse ways in which we wish to edit and refine their outputs. In this study, we broaden the scope of the editing problem to include an array of editing cases such as debiasing and rectifying reasoning errors and define an edit as any natural language expression that solicits a change in the model's outputs. We are introducing DUnE-an editing benchmark where edits are natural language sentences and propose that DUnE presents a challenging yet relevant task. To substantiate this claim, we conduct an extensive series of experiments testing various editing approaches to address DUnE, demonstrating their respective strengths and weaknesses. We show that retrieval-augmented language modeling can outperform specialized editing techniques and neither set of approaches has fully solved the generalized editing problem covered by our benchmark.

翻译：即便是最先进的语言模型仍难以避免错误，这迫使我们需要在不启动全面重新训练的情况下修改这些模型。模型编辑是指以产生预期结果的方式修改模型的知识或表征。先前的研究主要聚焦于编辑事实数据（例如“梅西为迈阿密国际效力”），将编辑的定义局限于知识三元组（即主体、客体、关系）。然而，随着语言模型应用范围的扩大，我们希望编辑和优化其输出的方式也日趋多样化。在本研究中，我们将编辑问题的范围拓展至包括去偏、修正推理错误等一系列编辑案例，并将编辑定义为任何请求改变模型输出的自然语言表达。我们提出了DUnE——一个以自然语言句子作为编辑形式的编辑基准，并认为DUnE呈现了一个既具挑战性又具相关性的任务。为佐证这一论断，我们开展了一系列广泛实验，测试了多种针对DUnE的编辑方法，展示了它们各自的优势与不足。研究表明，检索增强的语言模型能够超越专门的编辑技术，而这两类方法均未能完全解决我们基准所涵盖的广义编辑问题。