When the world changes, so does the text that humans write about it. How do we build language models that can be easily updated to reflect these changes? One popular approach is retrieval-augmented generation, in which new documents are inserted into a knowledge base and retrieved during prediction for downstream tasks. Most prior work on these systems have focused on improving behavior during prediction through better retrieval or reasoning. This paper introduces ERASE, which instead improves model behavior when new documents are acquired, by incrementally deleting or rewriting other entries in the knowledge base each time a document is added. In two new benchmark datasets evaluating models' ability to answer questions about a stream of news articles or conversations, ERASE improves accuracy relative to conventional retrieval-augmented generation by 7-13% (Mixtral-8x7B) and 6-10% (Llama-3-8B) absolute. Code and data are available at https://github.com/belindal/ERASE

翻译：当世界发生变化时，人类描述它的文本也随之改变。我们如何构建能够轻松更新以反映这些变化的语言模型？一种主流方法是检索增强生成，该方法将新文档插入知识库，并在下游任务预测过程中进行检索。以往关于此类系统的研究大多侧重于通过改进检索或推理机制来优化预测阶段的行为。本文提出的ERASE方法则另辟蹊径，它通过在每次添加文档时增量式删除或重写知识库中的其他条目，来提升模型在获取新文档时的表现。在两个新构建的基准数据集（用于评估模型对新闻流或对话流的问题回答能力）上的实验表明：相较于传统检索增强生成方法，ERASE在Mixtral-8x7B模型上实现了7-13%的绝对准确率提升，在Llama-3-8B模型上实现了6-10%的绝对准确率提升。代码与数据已发布于https://github.com/belindal/ERASE。