Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation. In this work, we show that this type of approach can be further improved with multilingual training. We observe significant improvements in zero-shot cross-modal speech translation, even outperforming a supervised approach based on XLSR for several languages.

翻译：近期研究表明，通过共享固定大小表征将独立训练的编码器和解码器相结合，可在语音到文本翻译中取得具有竞争力的性能。本工作表明，此类方法可通过多语言训练进一步优化。我们观察到该方法在零样本跨模态语音翻译中取得了显著提升，在多个语言上甚至优于基于XLSR的有监督方法。