Large language models (LLMs) like ChatGPT and GPT-4 have exhibited remarkable abilities on a wide range of natural language processing (NLP) tasks, including various machine translation abilities accomplished during chat. However, these models are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. Therefore, we propose the $\mathbf{ParroT}$ framework to enhance and regulate the translation abilities during chat based on open-sourced LLMs (i.e., LLaMA-7b, BLOOMZ-7b-mt) and human written translation and evaluation data. Specifically, ParroT reformulates translation data into the instruction-following style, and introduces a "$\mathbf{Hint}$" field for incorporating extra requirements to regulate the translation process. Accordingly, we propose three instruction types for finetuning ParroT models, including translation instruction, contrastive instruction, and error-guided instruction. We can finetune either the full models or partial parameters via low rank adaptation (LoRA). Experiments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs significantly while error-guided instruction can lead to a further improvement, which demonstrates the importance of learning from low-quality translations annotated by human. Meanwhile, the ParroT models can also preserve the ability on general tasks with the Alpaca multi-task dataset involved in finetuning. Please refer to our Github project for more implementation details: https://github.com/wxjiao/ParroT

翻译：诸如ChatGPT和GPT-4等大语言模型（LLMs）在广泛自然语言处理任务（包括对话中完成多种机器翻译能力）上展现出显著能力。然而，这些模型仅通过受限API可访问，这为领域内的新研究与进展设置了障碍。为此，我们提出$\mathbf{ParroT}$框架，旨在基于开源大语言模型（如LLaMA-7b、BLOOMZ-7b-mt）及人工编写的翻译与评估数据，增强并规范对话中的翻译能力。具体而言，ParroT将翻译数据重构为指令遵循风格，并引入“$\mathbf{Hint}$”字段以融入额外要求来调控翻译过程。据此，我们提出三类指令用于微调ParroT模型：翻译指令、对比指令和错误引导指令。我们可通过低秩适配（LoRA）微调完整模型或部分参数。在Flores子集和WMT22测试集上的实验表明，翻译指令显著提升基础大语言模型的翻译性能，而错误引导指令可带来进一步改进，这证明了从人工标注的低质量翻译中学习的重要性。与此同时，ParroT模型在涉及Alpaca多任务数据集微调时，仍能保持通用任务能力。更多实现细节请参见我们的GitHub项目：https://github.com/wxjiao/ParroT