Simultaneous speech translation requires translating source speech into a target language in real-time while handling non-monotonic word dependencies. Traditional approaches rely on supervised training with word-level aligned data, which is difficult to collect at scale and thus depends on synthetic alignments using language-specific heuristics that are suboptimal. We propose Hibiki-Zero, which eliminates the need for word-level alignments entirely. This fundamentally simplifies the training pipeline and enables seamless scaling to diverse languages with varying grammatical structures, removing the bottleneck of designing language-specific alignment heuristics. We first train on sentence-level aligned data to learn speech translation at high latency, then apply a novel reinforcement learning strategy using GRPO to optimize latency while preserving translation quality. Hibiki-Zero achieves state-of-the-art performance in translation accuracy, latency, voice transfer, and naturalness across five X-to-English tasks. Moreover, we demonstrate that our model can be adapted to support a new input language with less than 1000h of speech. We provide examples, model weights, inference code and we release a benchmark containing 45h of multilingual data for speech translation evaluation.

翻译：同步语音翻译要求在实时处理非单调词依赖关系的同时，将源语音翻译成目标语言。传统方法依赖于词级对齐数据的监督训练，此类数据难以大规模收集，因而需要依赖使用语言特定启发式方法生成的次优合成对齐。我们提出Hibiki-Zero模型，该模型完全消除了对词级对齐的需求。这从根本上简化了训练流程，并能无缝扩展到具有不同语法结构的多样化语言，消除了设计语言特定对齐启发式方法的瓶颈。我们首先在句子级对齐数据上进行训练以学习高延迟下的语音翻译，随后应用一种采用GRPO的新型强化学习策略来优化延迟，同时保持翻译质量。Hibiki-Zero在五项X到英语翻译任务中，在翻译准确度、延迟、音色迁移和自然度方面均达到最先进性能。此外，我们证明该模型可通过少于1000小时的语音数据适配支持新的输入语言。我们提供了示例、模型权重、推理代码，并发布了包含45小时多语言数据的语音翻译评估基准。