End-to-end simultaneous speech translation (SimulST) outputs translation while receiving the streaming speech inputs (a.k.a. streaming speech translation), and hence needs to segment the speech inputs and then translate based on the current received speech. However, segmenting the speech inputs at unfavorable moments can disrupt the acoustic integrity and adversely affect the performance of the translation model. Therefore, learning to segment the speech inputs at those moments that are beneficial for the translation model to produce high-quality translation is the key to SimulST. Existing SimulST methods, either using the fixed-length segmentation or external segmentation model, always separate segmentation from the underlying translation model, where the gap results in segmentation outcomes that are not necessarily beneficial for the translation process. In this paper, we propose Differentiable Segmentation (DiSeg) for SimulST to directly learn segmentation from the underlying translation model. DiSeg turns hard segmentation into differentiable through the proposed expectation training, enabling it to be jointly trained with the translation model and thereby learn translation-beneficial segmentation. Experimental results demonstrate that DiSeg achieves state-of-the-art performance and exhibits superior segmentation capability.

翻译：端到端同步语音翻译（SimulST）在接收流式语音输入的同时输出翻译结果（即流式语音翻译），因此需要对语音输入进行分段，并基于当前接收到的语音进行翻译。然而，在不合适的时机对语音输入进行分段会破坏声学完整性，并对翻译模型的性能产生不利影响。因此，学习在有利于翻译模型产生高质量翻译的时机对语音输入进行分段，是SimulST的关键。现有SimulST方法（无论是使用固定长度分段还是外部分段模型）总是将分段与底层翻译模型分离，这种分离导致分段结果未必有利于翻译过程。本文针对SimulST提出可微分分段（DiSeg）方法，使其能够直接从底层翻译模型中学习分段策略。DiSeg通过所提出的期望训练将硬分段转化为可微分操作，从而能够与翻译模型联合训练，进而学习到有利于翻译的分段方式。实验结果表明，DiSeg实现了最先进的性能，并展现出卓越的分段能力。