The training paradigm for machine translation has gradually shifted, from learning neural machine translation (NMT) models with extensive parallel corpora to instruction finetuning on pretrained multilingual large language models (LLMs) with high-quality translation pairs. In this paper, we focus on boosting the many-to-many multilingual translation performance of LLMs with an emphasis on zero-shot translation directions. We demonstrate that prompt strategies adopted during instruction finetuning are crucial to zero-shot translation performance and introduce a cross-lingual consistency regularization, XConST, to bridge the representation gap among different languages and improve zero-shot translation performance. XConST is not a new method, but a version of CrossConST (Gao et al., 2023a) adapted for multilingual finetuning on LLMs with translation instructions. Experimental results on ALMA (Xu et al., 2023) and LLaMA-2 (Touvron et al., 2023) show that our approach consistently improves translation performance. Our implementations are available at https://github.com/gpengzhi/CrossConST-LLM.

翻译：机器翻译的训练范式已逐渐从使用大规模平行语料库学习神经机器翻译（NMT）模型，转向基于预训练多语言大语言模型（LLM）结合高质量翻译对进行指令微调。本文聚焦于提升LLM在多对多多语言翻译中的性能，尤其关注零样本翻译方向。我们证明指令微调过程中采用的提示策略对零样本翻译性能至关重要，并引入跨语言一致性正则化方法XConST，以弥合不同语言间的表征差异并提升零样本翻译效果。XConST并非全新方法，而是将CrossConST（Gao等，2023a）适配至基于翻译指令的LLM多语言微调版本。在ALMA（Xu等，2023）与LLaMA-2（Touvron等，2023）上的实验结果表明，本方法能持续提升翻译性能。实现代码已开源至https://github.com/gpengzhi/CrossConST-LLM。