In this paper, we focus on editing Multimodal Large Language Models (MLLMs). Compared to editing single-modal LLMs, multimodal model editing is more challenging, which demands a higher level of scrutiny and careful consideration in the editing process. To facilitate research in this area, we construct a new benchmark, dubbed MMEdit, for editing multimodal LLMs and establishing a suite of innovative metrics for evaluation. We conduct comprehensive experiments involving various model editing baselines and analyze the impact of editing different components for multimodal LLMs. Empirically, we notice that previous baselines can implement editing multimodal LLMs to some extent, but the effect is still barely satisfactory, indicating the potential difficulty of this task. We hope that our work can provide the NLP community with insights. Code and dataset are available in https://github.com/zjunlp/EasyEdit.

翻译：本文聚焦于多模态大语言模型（MLLMs）的编辑任务。相较于编辑单模态语言模型，多模态模型编辑更具挑战性，要求编辑过程中进行更高层次的审慎考量和精细权衡。为促进该领域研究，我们构建了名为MMEdit的新基准，用于编辑多模态LLMs并建立了一套创新评估指标。我们开展了包含多种模型编辑基线的综合实验，系统分析了针对多模态LLMs不同组件进行编辑的影响。实证结果表明，先前基线方法虽能一定程度实现多模态LLMs的编辑，但效果仍差强人意，凸显该任务的潜在难度。期待本研究能为自然语言处理社区提供参考。代码与数据集已开源至https://github.com/zjunlp/EasyEdit。