Text-to-image models are trained on extensive amounts of data, leading them to implicitly encode factual knowledge within their parameters. While some facts are useful, others may be incorrect or become outdated (e.g., the current President of the United States). We introduce ReFACT, a novel approach for editing factual knowledge in text-to-image generative models. ReFACT updates the weights of a specific layer in the text encoder, only modifying a tiny portion of the model's parameters, and leaving the rest of the model unaffected. We empirically evaluate ReFACT on an existing benchmark, alongside RoAD, a newly curated dataset. ReFACT achieves superior performance in terms of generalization to related concepts while preserving unrelated concepts. Furthermore, ReFACT maintains image generation quality, making it a valuable tool for updating and correcting factual information in text-to-image models.

翻译：文本到图像模型在大量数据上进行训练，导致其隐式地将事实知识编码到参数中。虽然某些事实是有用的，但其他事实可能不正确或过时（例如，美国总统）。我们提出ReFACT，一种用于编辑文本到图像生成模型中事实知识的新方法。ReFACT更新文本编码器中特定层的权重，仅修改模型参数的微小部分，并保持模型其余部分不受影响。我们在现有基准测试以及新创建的RoAD数据集上对ReFACT进行经验评估。ReFACT在泛化到相关概念方面实现了优越性能，同时保持了无关概念不受影响。此外，ReFACT保持了图像生成质量，使其成为更新和纠正文本到图像模型中事实信息的宝贵工具。