Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of MLLM Lifelong Unlearning. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a large-scale and comprehensive benchmark featuring 127 entities across 9 classes under lifelong unlearning requests. We perform extensive experiments using MLUBench and reveal that existing unlearning methods suffer from severe, cumulative degradation. More critically, we further identify the unique challenge of this problem: unlike in unimodal models, MLLM lifelong unlearning is constrained by the need to preserve multimodal alignment. Continually unlearning from one modality could degrade the entire model. To alleviate this challenge, we propose LUMoE, an effective method. Experiments demonstrate that LUMoE significantly mitigates the degradation problem faced by baselines. The source code and the MLUBench dataset are open-sourced in https://github.com/lihe-maxsize/Lifelong_Unlearning_main.

翻译：多模态大语言模型（MLLMs）在海量多模态数据上训练，这使得数据遗忘变得日益重要——数据所有者可能要求移除特定内容。实际应用中，这类请求往往随时间顺序到达，由此催生了MLLM终身遗忘这一具有挑战性的难题。然而，现有基准测试在规模和覆盖范围上存在局限，未能充分捕捉MLLM终身遗忘的复杂性。为填补这一空白，我们提出了MLUBench——一个大规模、综合性的基准测试，包含127个实体、9个类别，覆盖终身遗忘请求场景。基于MLUBench开展的大量实验表明，现有遗忘方法存在严重且累积的性能退化。更关键的是，我们进一步揭示了该问题的独特挑战：与单模态模型不同，MLLM终身遗忘受限于多模态对齐的保持需求。持续对某一模态进行遗忘可能导致整个模型性能衰退。为缓解这一挑战，我们提出了有效方法LUMoE。实验证明，LUMoE显著缓解了基线方法面临的性能退化问题。源代码与MLUBench数据集已开源至 https://github.com/lihe-maxsize/Lifelong_Unlearning_main。