This paper explores SynTOD, a new synthetic data generation approach for developing end-to-end Task-Oriented Dialogue (TOD) Systems capable of handling complex tasks such as intent classification, slot filling, conversational question-answering, and retrieval-augmented response generation, without relying on crowdsourcing or real-world data. SynTOD utilizes a state transition graph to define the desired behavior of a TOD system and generates diverse, structured conversations through random walks and response simulation using large language models (LLMs). In our experiments, using graph-guided response simulations leads to significant improvements in intent classification, slot filling and response relevance compared to naive single-prompt simulated conversations. We also investigate the end-to-end TOD effectiveness of different base and instruction-tuned LLMs, with and without the constructed synthetic conversations. Finally, we explore how various LLMs can evaluate responses in a TOD system and how well they are correlated with human judgments. Our findings pave the path towards quick development and evaluation of domain-specific TOD systems. We release our datasets, models, and code for research purposes.

翻译：本文探索了SynTOD——一种新的合成数据生成方法，用于开发能够处理意图分类、槽位填充、对话式问答及检索增强响应生成等复杂任务的端到端面向任务对话系统，且无需依赖众包或真实世界数据。SynTOD利用状态转移图定义对话系统的预期行为，通过随机游走与大语言模型驱动的响应模拟生成多样化、结构化的对话。实验表明，与简单单提示模拟对话相比，采用图引导的响应模拟在意图分类、槽位填充及响应相关性方面均取得显著提升。我们进一步探究了不同基座模型及指令微调的大语言模型在有无构造的合成对话条件下的端到端面向任务对话效能。最后，我们检验了各类大语言模型评估对话系统响应的能力及其与人类判断的相关性。研究结果为快速开发与评估特定领域面向任务对话系统铺平了道路。为促进研究，我们公开了数据集、模型与代码。