ChatGPT shows remarkable capabilities for machine translation (MT). Several prior studies have shown that it achieves comparable results to commercial systems for high-resource languages, but lags behind in complex tasks, e.g, low-resource and distant-language-pairs translation. However, they usually adopt simple prompts which can not fully elicit the capability of ChatGPT. In this report, we aim to further mine ChatGPT's translation ability by revisiting several aspects: temperature, task information, and domain information, and correspondingly propose two (simple but effective) prompts: Task-Specific Prompts (TSP) and Domain-Specific Prompts (DSP). We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information further improves ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community. We also explore the effects of advanced in-context learning strategies and find a (negative but interesting) observation: the powerful chain-of-thought prompt leads to word-by-word translation behavior, thus bringing significant translation degradation.

翻译：ChatGPT展现出卓越的机器翻译能力。已有研究表明，在高资源语言翻译任务中，ChatGPT能达到与商业系统相当的水平，但在低资源语言对及远距离语言对翻译等复杂任务中仍存在差距。然而，这些研究通常采用简单提示词，未能充分挖掘ChatGPT的翻译能力。本报告旨在通过重新审视温度参数、任务信息和领域信息等要素，进一步挖掘ChatGPT的翻译能力，并相应提出两种（简单但有效）的提示策略：任务特定提示（TSP）与领域特定提示（DSP）。研究表明：1) ChatGPT的翻译性能高度依赖温度参数，较低温度通常能获得更优效果；2) 强化任务信息可进一步提升ChatGPT性能，尤其在复杂机器翻译任务中表现显著；3) 引入领域信息能激发ChatGPT的泛化能力，提升特定领域的翻译表现；4) 在非英语为中心的翻译任务中，ChatGPT易产生幻觉现象，本文提出的提示策略虽能部分缓解该问题，但仍需机器翻译/自然语言处理学界予以重点关注。此外，我们探索了高级上下文学习策略的影响，并得出（消极但有趣的）发现：强大的思维链提示会引发逐词翻译行为，导致翻译质量显著下降。