We propose a two-stage approach for training a single NMT model to translate unseen languages both to and from English. For the first stage, we initialize an encoder-decoder model to pretrained XLM-R and RoBERTa weights, then perform multilingual fine-tuning on parallel data in 40 languages to English. We find this model can generalize to zero-shot translations on unseen languages. For the second stage, we leverage this generalization ability to generate synthetic parallel data from monolingual datasets, then bidirectionally train with successive rounds of back-translation. Our approach, which we EcXTra (English-centric Crosslingual (X) Transfer), is conceptually simple, only using a standard cross-entropy objective throughout. It is also data-driven, sequentially leveraging auxiliary parallel data and monolingual data. We evaluate unsupervised NMT results for 7 low-resource languages, and find that each round of back-translation training further refines bidirectional performance. Our final single EcXTra-trained model achieves competitive translation performance in all translation directions, notably establishing a new state-of-the-art for English-to-Kazakh (22.9 > 10.4 BLEU). Our code is available at https://github.com/manestay/EcXTra .

翻译：我们提出了一种两阶段方法，用于训练单一NMT模型实现未知语言与英语之间的双向翻译。第一阶段，我们使用预训练的XLM-R和RoBERTa权重初始化编码器-解码器模型，然后在包含40种语言到英语的平行数据上进行多语言微调。研究发现该模型能够泛化至未知语言的零样本翻译任务。第二阶段，我们利用这种泛化能力从单语数据集中生成合成平行数据，随后通过连续多轮回译进行双向训练。我们提出的方法名为EcXTra（以英语为中心的跨语言迁移），其概念简洁，全程仅使用标准交叉熵目标函数；同时具备数据驱动特性，可依次利用辅助平行数据和单语数据。我们在7种低资源语言上评估了无监督NMT结果，发现每轮回译训练均能进一步提升双向翻译性能。最终单一EcXTra训练模型在所有翻译方向上均取得了有竞争力的性能，尤其在英语-哈萨克语方向上以22.9 > 10.4 BLEU值创下新纪录。代码已开源至https://github.com/manestay/EcXTra。