Fine-tuning is widely used to tailor large language models for specific tasks such as neural machine translation (NMT). However, leveraging transfer learning is computationally expensive when fine-tuning large multilingual models with billions of parameters, thus creating a barrier to entry for researchers working on low-resource domains such as Irish translation. Parameter-efficient fine-tuning (PEFT) bridges this gap by training on a fraction of the original model parameters, with the Low-Rank Adaptation (LoRA) approach introducing small, trainable adapter layers. We introduce SemiAdapt and SemiLoRA as semi-supervised inference-efficient approaches that strengthen domain adaptation and lead to improved overall performance in NMT. We demonstrate that SemiAdapt can outperform full-domain fine-tuning, while most notably, SemiLoRA can propel PEFT methods to match or even outperform full-model fine-tuning. We further evaluate domain-by-dataset fine-tuning and demonstrate that our embedding-based inference methods perform especially well on larger and noisier corpora. All Irish translation models developed in this work are released as open resources. These methods aim to make high-quality domain adaptation and fine-tuning more accessible to researchers working with low-resource languages.

翻译：微调技术被广泛用于针对神经机器翻译等特定任务定制大语言模型。然而，当对具有数十亿参数的大规模多语言模型进行微调时，利用迁移学习的计算成本极高，这为从事爱尔兰语翻译等低资源领域的研究者设置了准入障碍。参数高效微调方法通过仅训练原模型参数的一小部分来弥合这一差距，其中低秩自适应方法通过引入小型可训练适配器层实现。本文提出SemiAdapt与SemiLoRA两种半监督推理高效方法，它们能增强领域自适应能力并提升神经机器翻译的整体性能。实验表明，SemiAdapt能够超越全领域微调效果，而尤为重要的是，SemiLoRA能使参数高效微调方法达到甚至超越全模型微调的性能。我们进一步评估了按数据集分领域的微调策略，并证明基于嵌入的推理方法在更大规模且噪声较多的语料上表现尤为突出。本研究开发的所有爱尔兰语翻译模型均已作为开放资源发布。这些方法旨在为从事低资源语言研究的人员提供更便捷的高质量领域自适应与微调技术路径。