Task-oriented dialog (TOD) systems facilitate users in accomplishing complex, multi-turn tasks through natural language. While instruction-tuned large language models (LLMs) have demonstrated strong performance on a range of single-turn NLP tasks, they often struggle with reliable multi-turn task completion in TOD settings, particularly when generating API calls required to interact with external systems. To address this, we introduce RealTOD, a novel framework that improves LLM-based TOD systems through (1) prompt chaining and (2) fine-grained feedback. Prompt chaining enables zero-shot generalization to new domains by automatically synthesizing a schema-aligned in-context example for the target task. Fine-grained feedback verifies each generated API call against the domain schema, identifies specific errors, and provides targeted correction prompts. To evaluate task completion reliability, we introduce full API Call Accuracy as a robust metric, along with detailed sub-metrics to capture common failure modes. We conduct extensive experiments on the SGD and BiTOD benchmarks using four LLMs. RealTOD improves Full API accuracy, surpassing state-of-the-art AutoTOD by 37.10% on SGD and supervised learning-based baseline SimpleTOD by 10.32% on BiTOD. Human evaluations further confirm that LLMs integrated with RealTOD achieve superior task completion, fluency, and informativeness compared to existing methods.

翻译：任务导向对话系统通过自然语言协助用户完成复杂的多轮任务。尽管经过指令微调的大型语言模型在众多单轮自然语言处理任务中展现出卓越性能，但在任务导向对话场景下实现可靠的多轮任务完成仍面临挑战，特别是在生成与外部系统交互所需的API调用时。为此，我们提出RealTOD创新框架，通过（1）提示链与（2）细粒度反馈两大机制改进基于大型语言模型的任务导向对话系统。提示链通过为目标任务自动合成模式对齐的上下文示例，实现对新领域的零样本泛化。细粒度反馈则依据领域模式验证每个生成的API调用，识别具体错误并提供针对性修正提示。为评估任务完成可靠性，我们引入完整API调用准确率作为鲁棒性评价指标，并辅以详细子指标捕捉常见失败模式。我们在SGD与BiTOD基准上使用四种大型语言模型进行广泛实验。RealTOD显著提升了完整API准确率：在SGD数据集上超越当前最优的AutoTOD方法37.10%，在BiTOD数据集上超越基于监督学习的基线方法SimpleTOD 10.32%。人工评估进一步证实，集成RealTOD的大型语言模型在任务完成度、流畅性与信息完整性方面均优于现有方法。