Spoken dialogue systems powered by large language models have demonstrated remarkable abilities in understanding human speech and generating appropriate spoken responses. However, these systems struggle with end-turn detection (ETD) -- the ability to distinguish between user turn completion and hesitation. This limitation often leads to premature or delayed responses, disrupting the flow of spoken conversations. In this paper, we introduce the ETD Dataset, the first public dataset for end-turn detection. The ETD dataset consists of both synthetic speech data generated with text-to-speech models and real-world speech data collected from web sources. We also propose SpeculativeETD, a novel collaborative inference framework that balances efficiency and accuracy to improve real-time ETD in resource-constrained environments. Our approach jointly employs a lightweight GRU-based model, which rapidly detects the non-speaking units in real-time on local devices, and a high-performance Wav2vec-based model running on the server to make a more challenging classification of distinguishing turn ends from mere pauses. Experiments demonstrate that the proposed SpeculativeETD significantly improves ETD accuracy while keeping the required computations low. Datasets and code will be available after the review.

翻译：基于大语言模型的语音对话系统在理解人类语音和生成适当语音响应方面展现了卓越能力。然而，这些系统在端轮次检测方面存在困难——即区分用户轮次完成与犹豫的能力。这一局限常导致系统响应过早或延迟，从而破坏语音对话的流畅性。本文中，我们介绍了ETD数据集，这是首个面向端轮次检测的公开数据集。ETD数据集包含通过文本转语音模型生成的合成语音数据，以及从网络来源收集的真实世界语音数据。我们还提出了SpeculativeETD，一种新颖的协同推理框架，该框架在效率与准确性之间取得平衡，以提升资源受限环境中的实时ETD性能。我们的方法联合使用了一个轻量级的基于GRU的模型（在本地设备上实时快速检测非语音单元）和一个运行在服务器上的基于Wav2vec的高性能模型（用于执行更具挑战性的区分轮次结束与短暂停顿的分类任务）。实验表明，所提出的SpeculativeETD在保持低计算需求的同时，显著提高了ETD的准确性。数据集和代码将在评审后公开。