Prior work has shown that large language models (LLMs) can translate unseen or low-resource languages by undergoing continued training or even by encoding a grammar book in their context. However, both methods typically overfit specific languages, with limited zero-shot transfer at test time. To translate extremely low-resource languages at scale, we argue that LLMs must acquire the meta-skill of utilizing in-context linguistic knowledge rather than memorizing specific languages. In this paper, we propose a reinforcement learning (RL) approach to unseen language translation given rich linguistic context, using a surface-level translation metric (chrF) as the reward. Empirically, despite the lightweight reward, our RL-trained models effectively extract and apply relevant linguistic information from the provided context, leading to better translations on completely unseen languages than in-context learning or supervised fine-tuning. Our analyses suggest that outcome-based RL can extend beyond conventional reasoning tasks like math and coding to serve as a recipe for language learning from context.

翻译：先前研究表明，大型语言模型通过持续训练甚至将语法书编码至上下文，能够翻译未见或低资源语言。然而，这两种方法通常过度拟合特定语言，在测试时零样本迁移能力有限。为实现大规模超低资源语言翻译，我们认为大型语言模型必须掌握利用上下文语言知识的元技能，而非死记硬背特定语言。本文提出一种基于强化学习的方法，在丰富语言语境下进行未见语言翻译，以表面层面翻译质量指标（chrF）作为奖励函数。实验表明，尽管采用轻量级奖励机制，经强化学习训练的模型仍能有效从给定语境中提取并应用相关语言信息，在完全未见语言上取得优于上下文学习或监督微调的翻译效果。我们的分析显示，基于结果的强化学习可超越数学、编程等传统推理任务，成为从语境中学习语言的有效范式。