Most of the recent work in leveraging Large Language Models (LLMs) such as GPT-3 for Machine Translation (MT) has focused on selecting the few-shot samples for prompting. In this work, we try to better understand the role of demonstration attributes for the in-context learning of translations through perturbations of high-quality, in-domain demonstrations. We find that asymmetric perturbation of the source-target mappings yield vastly different results. We show that the perturbation of the source side has surprisingly little impact, while target perturbation can drastically reduce translation quality, suggesting that it is the output text distribution that provides the most important learning signal during in-context learning of translations. We propose a method named Zero-Shot-Context to add this signal automatically in Zero-Shot prompting. We demonstrate that it improves upon the zero-shot translation performance of GPT-3, even making it competitive with few-shot prompted translations.

翻译：近年来，利用GPT-3等大型语言模型进行机器翻译的研究大多聚焦于选取少量样本进行提示。本研究通过扰动高质量领域内示例，试图更深入理解演示属性在翻译上下文学习中的作用。我们发现，源语言-目标语言映射的非对称扰动会产生截然不同的结果。源语言端扰动影响极小，而目标语言端扰动则显著降低翻译质量，这表明在翻译的上下文学习中，输出文本分布提供了最重要的学习信号。我们提出名为Zero-Shot-Context的方法，可在零样本提示中自动加入该信号。实验证明，该方法能提升GPT-3的零样本翻译性能，甚至使其与少样本提示翻译相媲美。