Large language models (LLMs) that have been trained on multilingual but not parallel text exhibit a remarkable ability to translate between languages. We probe this ability in an in-depth study of the pathways language model (PaLM), which has demonstrated the strongest machine translation (MT) performance among similarly-trained LLMs to date. We investigate various strategies for choosing translation examples for few-shot prompting, concluding that example quality is the most important factor. Using optimized prompts, we revisit previous assessments of PaLM's MT capabilities with more recent test sets, modern MT metrics, and human evaluation, and find that its performance, while impressive, still lags that of state-of-the-art supervised systems. We conclude by providing an analysis of PaLM's MT output which reveals some interesting properties and prospects for future work.

翻译：基于多语言但非平行文本训练的大型语言模型展现出惊人的跨语言翻译能力。我们通过深度研究路径语言模型(PaLM)来探索这一能力——该模型在当前同类训练的大语言模型中展现出最强的机器翻译性能。通过设计少样本提示中翻译范例的选取策略，我们证实范例质量是关键影响因素。利用优化后的提示，我们使用最新测试集、现代机器翻译评测指标及人工评估重新审视了PaLM的翻译能力，发现其表现虽令人瞩目，但仍落后于当前最优监督系统。最后通过对PaLM翻译输出的分析，揭示了若干值得关注的特性与未来研究方向。