Task-oriented dialogue (TOD) models have made significant progress in recent years. However, previous studies primarily focus on datasets written by annotators, which has resulted in a gap between academic research and real-world spoken conversation scenarios. While several small-scale spoken TOD datasets are proposed to address robustness issues such as ASR errors, they ignore the unique challenges in spoken conversation. To tackle the limitations, we introduce SpokenWOZ, a large-scale speech-text dataset for spoken TOD, containing 8 domains, 203k turns, 5.7k dialogues and 249 hours of audios from human-to-human spoken conversations. SpokenWOZ further incorporates common spoken characteristics such as word-by-word processing and reasoning in spoken language. Based on these characteristics, we present cross-turn slot and reasoning slot detection as new challenges. We conduct experiments on various baselines, including text-modal models, newly proposed dual-modal models, and LLMs, e.g., ChatGPT. The results show that the current models still have substantial room for improvement in spoken conversation, where the most advanced dialogue state tracker only achieves 25.65% in joint goal accuracy and the SOTA end-to-end model only correctly completes the user request in 52.1% of dialogues. The dataset, code, and leaderboard are available: https://spokenwoz.github.io/.

翻译：任务型对话（TOD）模型近年来取得了显著进展。然而，先前研究主要关注由标注者编写的数据集，导致学术研究与真实口语对话场景之间存在差距。尽管已有若干小规模口语TOD数据集被提出以解决ASR错误等鲁棒性问题，但这些数据集忽视了口语对话中的独特挑战。为解决上述局限性，我们提出了SpokenWOZ——一个面向口语TOD的大规模语音文本数据集，包含来自人人对话的8个领域、20.3万轮次、5700段对话以及249小时音频。SpokenWOZ进一步融入了口语对话的常见特征，例如逐词处理及口语推理。基于这些特征，我们提出了跨轮槽位检测和推理槽位检测作为新挑战。我们在多种基线模型上开展了实验，包括文本模态模型、新提出的双模态模型以及大语言模型（如ChatGPT）。结果表明，当前模型在口语对话中仍有较大改进空间：最先进的对话状态追踪器联合目标准确率仅达25.65%，而最先进的端到端模型仅在52.1%的对话中正确完成用户请求。数据集、代码及排行榜均可在https://spokenwoz.github.io/获取。