Large language models show compelling performance on reasoning tasks but they tend to perform much worse in languages other than English. This is unsurprising given that their training data largely consists of English text and instructions. A typical solution is to translate instruction data into all languages of interest, and then train on the resulting multilingual data, which is called translate-training. This approach not only incurs high cost, but also results in poorly translated data due to the non-standard formatting of mathematical chain-of-thought. In this paper, we explore the benefits of question alignment, where we train the model to translate reasoning questions into English by finetuning on X-English parallel question data. In this way we perform targeted, in-domain language alignment which makes best use of English instruction data to unlock the LLMs' multilingual reasoning abilities. Experimental results on LLaMA2-13B show that question alignment leads to consistent improvements over the translate-training approach: an average improvement of 11.3% and 16.1% accuracy across ten languages on the MGSM and MSVAMP multilingual reasoning benchmarks. The project will be available at: https://github.com/NJUNLP/QAlign.

翻译：大型语言模型在推理任务上展现出引人注目的性能，但其在英语之外的其他语言中表现往往显著下降。鉴于其训练数据主要由英文文本和指令构成，这一现象并不令人意外。典型解决方案是将指令数据翻译成所有目标语言，然后在生成的多语言数据上进行训练，这种方法被称为翻译训练。该方案不仅成本高昂，而且由于数学思维链的非标准化格式，常导致翻译质量欠佳。本文探索了问题对齐的优势：通过基于X-英语平行问题数据进行微调，训练模型将推理问题翻译为英语。这种方法实现了针对性的领域内语言对齐，能够最大限度地利用英语指令数据来释放大型语言模型的多语言推理能力。在LLaMA2-13B上的实验结果表明，问题对齐方法相较翻译训练方案取得了持续改进：在MGSM和MSVAMP多语言推理基准测试的十种语言上，平均准确率分别提升11.3%和16.1%。本项目代码已发布于：https://github.com/NJUNLP/QAlign。