Multilingual transfer ability, which reflects how well the models fine-tuned on one source language can be applied to other languages, has been well studied in multilingual pre-trained models (e.g., BLOOM). However, such ability has not been investigated for English-centric models (e.g., LLaMA). To fill this gap, we study the following research questions. First, does multilingual transfer ability exist in English-centric models and how does it compare with multilingual pretrained models? Second, does it only appears when English is the source language for the English-centric model? Third, how does it vary in different tasks? We take multilingual reasoning ability as our focus and conduct extensive experiments across four types of reasoning tasks. We find that the multilingual pretrained model does not always outperform an English-centric model. Furthermore, English appears to be a less suitable source language, and the choice of source language becomes less important when the English-centric model scales up. In addition, different types of tasks exhibit different multilingual transfer abilities. These findings demonstrate that English-centric models not only possess multilingual transfer ability but may even surpass the transferability of multilingual pretrained models if well-trained. By showing the strength and weaknesses, the experiments also provide valuable insights into enhancing multilingual reasoning abilities for the English-centric models.

翻译：多语言迁移能力，即基于源语言微调的模型可应用于其他语言的程度，已在多语言预训练模型（如BLOOM）中得到广泛研究。然而，这种能力尚未在以英语为中心的模型（如LLaMA）中得到探究。为填补这一空白，我们研究了以下问题：第一，以英语为中心的模型是否存在多语言迁移能力？其表现如何与多语言预训练模型相比？第二，这种能力是否仅在以英语为源语言时出现？第三，不同任务中该能力如何变化？我们以多语言推理能力为核心，在四类推理任务上开展了广泛实验。研究发现，多语言预训练模型并不总是优于以英语为中心的模型。此外，英语作为源语言时表现欠佳，且随着以英语为中心的模型规模扩大，源语言的选择变得不再重要。同时，不同类型任务展现出不同的多语言迁移能力。这些结果表明，以英语为中心的模型不仅具备多语言迁移能力，而且若经过充分训练，其迁移能力甚至可能超越多语言预训练模型。通过揭示优势与不足，该实验亦为增强以英语为中心模型的多语言推理能力提供了宝贵见解。