Large language models (LLMs) like ChatGPT and GPT-4 have exhibited remarkable abilities on a wide range of natural language processing (NLP) tasks, including various machine translation abilities accomplished during chat. However, these models are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. Therefore, we propose the $\mathbf{ParroT}$ framework to enhance and regulate the translation abilities during chat based on open-sourced LLMs (i.e., LLaMA-7b, BLOOMZ-7b-mt) and human written translation and evaluation data. Specifically, ParroT reformulates translation data into the instruction-following style, and introduces a "$\mathbf{Hint}$" field for incorporating extra requirements to regulate the translation process. Accordingly, we propose three instruction types for finetuning ParroT models, including translation instruction, contrastive instruction, and error-guided instruction. We can finetune either the full models or partial parameters via low rank adaptation (LoRA). Experiments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs significantly while error-guided instruction can lead to a further improvement, which demonstrates the importance of learning from low-quality translations annotated by human. Meanwhile, the ParroT models can also preserve the ability on general tasks with the Alpaca multi-task dataset involved in finetuning. Please refer to our Github project for more implementation details: https://github.com/wxjiao/ParroT

翻译：大型语言模型（LLMs）如ChatGPT和GPT-4在广泛的自然语言处理（NLP）任务中展现出卓越能力，包括在聊天过程中完成的多种机器翻译能力。然而，这些模型仅通过受限API可访问，这为该领域的新研究和进展设置了障碍。因此，我们提出$\mathbf{ParroT}$框架，基于开源LLMs（即LLaMA-7b、BLOOMZ-7b-mt）及人工撰写的翻译与评估数据，增强并规范聊天中的翻译能力。具体而言，ParroT将翻译数据重新表述为指令遵循风格，并引入“$\mathbf{Hint}$”字段以纳入额外要求来规范翻译过程。相应地，我们提出三种指令类型用于微调ParroT模型，包括翻译指令、对比指令和错误引导指令。我们可通过低秩适应（LoRA）微调完整模型或部分参数。在Flores子集和WMT22测试集上的实验表明，翻译指令显著提升了原始LLMs的翻译性能，而错误引导指令可带来进一步改进，这证明了从人工标注的低质量翻译中学习的重要性。同时，通过将Alpaca多任务数据集纳入微调，ParroT模型还能保持对通用任务的处理能力。更多实现细节请参考我们的Github项目：https://github.com/wxjiao/ParroT