Large language models (LLMs) have made significant progress in machine translation (MT). However, their potential in domain-specific MT remains under-explored. Current LLM-based MT systems still face several challenges. First, for LLMs with in-context learning, their effectiveness is highly sensitive to input translation examples, and processing them can increase inference costs. They often require extra post-processing due to over-generation. Second, LLMs with fine-tuning on domain-specific data often require high training costs for domain adaptation, and may weaken the zero-shot MT capabilities of LLMs due to over-specialization. The aforementioned methods can struggle to translate rare words in domain transfer scenarios. To address these challenges, this paper proposes a prompt-oriented fine-tuning method, denoted as LlamaIT, to effectively and efficiently fine-tune a general-purpose LLM for domain-specific MT tasks. First, we construct a task-specific mix-domain dataset, which is then used to fine-tune the LLM with LoRA. This can eliminate the need for input translation examples, post-processing, or over-specialization. By zero-shot prompting with instructions, we adapt the MT tasks to the target domain at inference time. To further elicit the MT capability for rare words, we construct new prompts by incorporating domain-specific bilingual vocabulary. We also conduct extensive experiments on both publicly available and self-constructed datasets. The results show that our LlamaIT can significantly enhance the domain-specific MT capabilities of the LLM, meanwhile preserving its zero-shot MT capabilities.

翻译：大语言模型（LLM）在机器翻译（MT）领域已取得显著进展。然而，它们在特定领域机器翻译中的潜力仍待深入挖掘。当前基于LLM的MT系统仍面临若干挑战。首先，对于采用上下文学习的LLM，其效果对输入翻译样例高度敏感，且处理这些样例可能增加推理成本。由于存在过度生成问题，通常需要额外后处理。其次，通过领域特定数据进行微调的LLM，领域适配需要高昂训练成本，且可能因过度专业化而削弱LLM的零样本MT能力。上述方法在领域迁移场景中难以处理罕见词。针对这些挑战，本文提出一种面向提示的微调方法（记为LlamaIT），用于高效地将通用LLM适配至特定领域MT任务。首先构建任务特定的混合领域数据集，采用LoRA对该数据集进行LLM微调。该方法无需输入翻译样例、后处理或处理过度专业化问题。通过零样本指令提示，在推理阶段将MT任务适配至目标领域。为进一步激发罕见词的MT能力，我们通过融入领域特定的双语词汇表构建新提示。在公开数据集和自建数据集上进行的广泛实验表明，LlamaIT能显著增强LLM的特定领域MT能力，同时保持其零样本MT能力。