Task-oriented dialogue (TOD) models have made significant progress in recent years. However, previous studies primarily focus on datasets written by annotators, which has resulted in a gap between academic research and real-world spoken conversation scenarios. While several small-scale spoken TOD datasets are proposed to address robustness issues such as ASR errors, they ignore the unique challenges in spoken conversation. To tackle the limitations, we introduce SpokenWOZ, a large-scale speech-text dataset for spoken TOD, containing 8 domains, 203k turns, 5.7k dialogues and 249 hours of audios from human-to-human spoken conversations. SpokenWOZ further incorporates common spoken characteristics such as word-by-word processing and reasoning in spoken language. Based on these characteristics, we present cross-turn slot and reasoning slot detection as new challenges. We conduct experiments on various baselines, including text-modal models, newly proposed dual-modal models, and LLMs, e.g., ChatGPT. The results show that the current models still have substantial room for improvement in spoken conversation, where the most advanced dialogue state tracker only achieves 25.65% in joint goal accuracy and the SOTA end-to-end model only correctly completes the user request in 52.1% of dialogues. The dataset, code, and leaderboard are available: https://spokenwoz.github.io/SpokenWOZ-github.io/.

翻译：任务型对话（TOD）模型近年来取得了显著进展，但以往研究主要聚焦于标注者编写的书面数据集，导致学术研究与实际口语对话场景之间存在差距。尽管已有若干小规模口语TOD数据集被提出用于处理ASR错误等鲁棒性问题，但这些研究忽略了口语对话中的独特挑战。为突破上述局限，我们提出SpokenWOZ——面向口语TOD的大规模语音文本数据集，包含来自人际口语对话的8个领域、203k轮次、5.7k段对话及249小时音频。SpokenWOZ进一步融入了口语常见特征，如逐词处理及口语推理。基于这些特征，我们提出跨轮次槽位检测和推理槽位检测作为新挑战。我们在多种基准模型上开展实验，包括文本模态模型、新提出的双模态模型以及ChatGPT等大型语言模型。结果表明，现有模型在口语对话场景中仍有显著提升空间：最先进的对话状态追踪器联合目标准确率仅达25.65%，端到端模型的最优方案仅在52.1%的对话中正确完成用户请求。数据集、代码及排行榜已公开：https://spokenwoz.github.io/SpokenWOZ-github.io/。