Large-scale Pretrained Language Models (LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language, the performance depends on its similarity to English and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instructions and the alignment among different languages. With multilingual finetuning, LLMs could learn to perform the translation task well even for those language pairs unseen during the instruction tuning phase.

翻译：大规模预训练语言模型（LLMs），如ChatGPT和GPT4，在未经过平行语料显式训练的情况下，已展现出强大的多语言翻译能力。这些LLMs如何获得执行不同语言翻译指令的能力，是一个值得探究的问题。本文通过微调多语言预训练语言模型XGLM-7B，使其能够遵循给定指令执行多语言翻译，并进行了详细分析。首先，我们发现多语言LLMs的翻译能力比此前展示的更强。对于特定语言，其性能取决于该语言与英语的相似度以及预训练阶段使用的数据量。其次，我们发现LLMs执行翻译指令的能力依赖于对翻译指令的理解以及不同语言之间的对齐。通过多语言微调，LLMs能够学会良好地执行翻译任务，即使对于指令微调阶段未见过的语言对也是如此。