Large language models (LLMs) demonstrate promising translation performance among various natural languages. However, many LLMs especially the open-sourced ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of natural languages, making the potential of LLMs on language translation less explored. In this work, we present BigTranslate which adapts LLaMA that covers only 20 languages and enhances it with multilingual translation capability on more than 100 languages. BigTranslate is built upon LLaMA-13B and it is optimized in three steps. First, we continue training LLaMA with massive Chinese monolingual data. Second, we continue training the model with a large-scale parallel dataset that covers 102 natural languages. Third, we instruct-tune the foundation model with multilingual translation instructions, leading to our BigTranslate model. The preliminary experiments on multilingual translation show that BigTranslate performs comparably with ChatGPT and Google Translate in many languages and even outperforms ChatGPT in 8 language pairs. We release the BigTranslate model and hope it can advance the research progress.

翻译：大型语言模型（LLMs）在各种自然语言之间展现了有前景的翻译性能。然而，许多LLMs，尤其是开源模型（如BLOOM和LLaMA），以英语为主导语言，仅支持数十种自然语言，使得LLMs在语言翻译方面的潜力未能得到充分探索。本文提出BigTranslate，该方法对仅覆盖20种语言的LLaMA进行适配，并赋予其针对超过100种语言的多语言翻译能力。BigTranslate基于LLaMA-13B构建，并通过三个步骤进行优化。首先，我们使用大规模中文单语数据对LLaMA进行持续训练。其次，使用覆盖102种自然语言的大规模平行数据集对模型进行持续训练。最后，通过多语言翻译指令对基础模型进行指令微调，从而得到BigTranslate模型。多语言翻译的初步实验表明，BigTranslate在许多语言上的表现与ChatGPT和谷歌翻译相当，甚至在8个语言对中优于ChatGPT。我们开源了BigTranslate模型，希望推动相关研究进展。