Research in multilingual speech-to-text translation is topical. Having a single model that supports multiple translation tasks is desirable. The goal of this work it to improve cross-lingual transfer learning in multilingual speech-to-text translation via semantic knowledge distillation. We show that by initializing the encoder of the encoder-decoder sequence-to-sequence translation model with SAMU-XLS-R, a multilingual speech transformer encoder trained using multi-modal (speech-text) semantic knowledge distillation, we achieve significantly better cross-lingual task knowledge transfer than the baseline XLS-R, a multilingual speech transformer encoder trained via self-supervised learning. We demonstrate the effectiveness of our approach on two popular datasets, namely, CoVoST-2 and Europarl. On the 21 translation tasks of the CoVoST-2 benchmark, we achieve an average improvement of 12.8 BLEU points over the baselines. In the zero-shot translation scenario, we achieve an average gain of 18.8 and 11.9 average BLEU points on unseen medium and low-resource languages. We make similar observations on Europarl speech translation benchmark.

翻译：多语言语音到文本翻译的研究是当前的热点方向。支持多种翻译任务的单一模型具有显著优势。本研究旨在通过语义知识蒸馏改进多语言语音到文本翻译中的跨语言迁移学习。我们证明，使用基于多模态（语音-文本）语义知识蒸馏训练的多语言语音Transformer编码器SAMU-XLS-R来初始化编码器-解码器序列到序列翻译模型的编码器，相比基线模型XLS-R（通过自监督学习训练的多语言语音Transformer编码器），能显著提升跨语言任务知识迁移效果。我们在两个广泛使用的数据集CoVoST-2和Europarl上验证了该方法的有效性。在CoVoST-2基准测试的21个翻译任务中，我们的方法相比基线平均提升了12.8个BLEU点。在零样本翻译场景下，对于未见过的中资源语言和低资源语言，分别实现了平均18.8和11.9个BLEU点的提升。在Europarl语音翻译基准测试中也观察到类似结果。