NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS) models flounder when sufficient amounts of parallel data is not available for fine-tuning. This specifically holds for languages missing/under-represented in these models. The problem gets aggravated when the data comes from different domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of PMSS models is extremely beneficial for domain-specific NMT, especially when target domain data is limited/unavailable and the considered languages are missing or under-represented in the PMSS model. We quantify the domain-specific results variations using a domain-divergence test, and show that ITFT can mitigate the impact of domain divergence to some extent.

翻译：基于预训练多语言序列到序列（PMSS）模型的神经机器翻译（NMT）系统，在缺乏足够平行数据进行微调时性能不佳，尤其当目标语言在模型中被缺失或低资源表达时更为突出。若数据来自不同领域，该问题会进一步加剧。本文证明，对PMSS模型进行中间任务微调（ITFT）对于领域特定的NMT极为有效，特别是在目标领域数据有限或不可用、且目标语言在PMSS模型中缺失或低资源表达的情况下。我们通过领域散度测试量化了领域特定的结果差异，并表明ITFT能在一定程度上缓解领域散度的影响。