Phrases are essential to understand the core concepts in conversations. However, due to their rare occurrence in training data, correct translation of phrases is challenging in speech translation tasks. In this paper, we propose a phrase dictionary biasing method to leverage pairs of phrases mapping from the source language to the target language. We apply the phrase dictionary biasing method to two types of widely adopted models, a transducer-based streaming speech translation model and a multimodal large language model. Experimental results show that the phrase dictionary biasing method outperforms phrase list biasing by 21% relatively for the streaming speech translation model. In addition, phrase dictionary biasing enables multimodal large language models to use external phrase information, achieving 85% relative improvement in phrase recall.
翻译:短语对于理解对话中的核心概念至关重要。然而,由于其在训练数据中出现频率较低,在语音翻译任务中正确翻译短语具有挑战性。本文提出一种短语词典偏置方法,以利用从源语言到目标语言的短语映射对。我们将该短语词典偏置方法应用于两类广泛采用的模型:基于Transducer的流式语音翻译模型和多模态大语言模型。实验结果表明,对于流式语音翻译模型,短语词典偏置方法相较于短语列表偏置实现了21%的相对性能提升。此外,短语词典偏置使多模态大语言模型能够利用外部短语信息,在短语召回率上实现了85%的相对改进。