Fine-tuning multilingual foundation models on specific languages often induces catastrophic forgetting, degrading performance on languages unseen in fine-tuning. While this phenomenon is widely-documented, the literature presents fragmented results about when forgetting occurs. To address this ambiguity, we conduct a systematic empirical study using machine translation as a testbed to identify the conditions that trigger catastrophic forgetting in multilingual fine-tuning. Through controlled experiments across different model architectures, data scales, and fine-tuning approaches, we reveal that the relative scale between model and data size is a primary determinant of forgetting. Moreover, we demonstrate that a model's instruction-following ability is more critical for retaining multilingual knowledge than its architecture. Contrary to assumptions, parameter-efficient fine-tuning offers no clear advantage over full fine-tuning in mitigating forgetting. Lastly, we show that cross-lingual alignment can mitigate forgetting while also facilitating positive transfer to unseen target languages.

翻译：在特定语言上对多语言基础模型进行微调常常会引发灾难性遗忘，导致在微调未见语言上的性能下降。尽管这一现象已被广泛记录，但现有文献对于遗忘何时发生给出了零散且不一致的结论。为厘清这一模糊性，我们以机器翻译为测试平台，开展了一项系统的实证研究，旨在识别在多语言微调中触发灾难性遗忘的条件。通过对不同模型架构、数据规模和微调方法进行对照实验，我们发现模型规模与数据规模之间的相对比例是决定遗忘的主要因素。此外，我们证明模型的指令遵循能力对于保留多语言知识比其架构更为关键。与通常的假设相反，参数高效微调在缓解遗忘方面并未显示出优于全参数微调的明显优势。最后，我们表明跨语言对齐不仅可以缓解遗忘，还能促进向未见目标语言的积极知识迁移。