Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations. We develop MEMIT, a method for directly updating a language model with many memories, demonstrating experimentally that it can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by orders of magnitude. Our code and data are at https://memit.baulab.info.

翻译：近期研究在更新大型语言模型以注入新记忆方面展现出令人振奋的前景，可用于替换过时信息或添加专业知识。然而，现有工作主要局限于更新单一关联。我们提出MEMIT方法，支持直接向语言模型注入大量记忆，实验证明该方法可扩展至GPT-J（6B）和GPT-NeoX（20B）模型数千个关联的更新，其规模较之前工作提升数个数量级。代码与数据已开源至https://memit.baulab.info。