Task-oriented dialogue (TOD) models have made significant progress in recent years. However, previous studies primarily focus on datasets written by annotators, which has resulted in a gap between academic research and real-world spoken conversation scenarios. While several small-scale spoken TOD datasets are proposed to address robustness issues such as ASR errors, they ignore the unique challenges in spoken conversation. To tackle the limitations, we introduce SpokenWOZ, a large-scale speech-text dataset for spoken TOD, containing 8 domains, 203k turns, 5.7k dialogues and 249 hours of audios from human-to-human spoken conversations. SpokenWOZ further incorporates common spoken characteristics such as word-by-word processing and reasoning in spoken language. Based on these characteristics, we present cross-turn slot and reasoning slot detection as new challenges. We conduct experiments on various baselines, including text-modal models, newly proposed dual-modal models, and LLMs, e.g., ChatGPT. The results show that the current models still have substantial room for improvement in spoken conversation, where the most advanced dialogue state tracker only achieves 25.65% in joint goal accuracy and the SOTA end-to-end model only correctly completes the user request in 52.1% of dialogues. The dataset, code, and leaderboard are available: https://spokenwoz.github.io/SpokenWOZ-github.io/.

翻译：任务型对话模型近年来取得了显著进展。然而，先前研究主要集中于标注员编写的书面数据集，导致学术研究与真实口语对话场景之间存在差距。尽管已提出若干小规模口语任务型对话数据集以解决ASR误差等鲁棒性问题，但此类研究忽视了口语对话中的独特挑战。为突破上述局限，我们提出SpokenWOZ——面向口语任务型对话的大规模语音-文本数据集，包含8个领域、20.3万轮次、5700段对话及249小时人类口语对话音频。SpokenWOZ进一步融入了逐词加工与口语推理等常见口语特征。基于这些特征，我们提出跨轮槽位检测与推理槽位检测作为新挑战。我们在多种基线模型上开展实验，包括纯文本模态模型、新型双模态模型及大语言模型（如ChatGPT）。结果表明，当前模型在口语对话方面仍有显著提升空间：最先进的对话状态追踪器联合目标准确率仅为25.65%，最先进的端到端模型仅在52.1%的对话中正确完成用户请求。数据集、代码及排行榜详见：https://spokenwoz.github.io/SpokenWOZ-github.io/。