NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS) models flounder when sufficient amounts of parallel data is not available for fine-tuning. This specifically holds for languages missing/under-represented in these models. The problem gets aggravated when the data comes from different domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of PMSS models is extremely beneficial for domain-specific NMT, especially when target domain data is limited/unavailable and the considered languages are missing or under-represented in the PMSS model. We quantify the domain-specific results variations using a domain-divergence test, and show that ITFT can mitigate the impact of domain divergence to some extent.

翻译：基于预训练多语言序列到序列（PMSS）模型训练的神经机器翻译（NMT）系统，在缺乏足够平行数据进行微调时表现不佳，尤其针对这些模型中缺失或代表性不足的语言。当数据来自不同领域时，问题进一步加剧。本文表明，对PMSS模型进行中间任务微调（ITFT）对领域特定的NMT极为有益，尤其是当目标领域数据有限或不可用，且考虑的语言在PMSS模型中缺失或代表性不足时。我们通过领域分歧测试量化了领域特定的结果变化，并证明ITFT能在一定程度上缓解领域分歧的影响。