During task-oriented dialogues (TODs), human users naturally introduce chitchat that is beyond the immediate scope of the task, interfering with the flow of the conversation. To address this issue without the need for expensive manual data creation, we use few-shot prompting with Llama-2-70B to enhance the MultiWOZ dataset with user backstories, a typical example of chitchat interference in TODs. We assess the impact of this addition by testing two models: one trained solely on TODs and another trained on TODs with a preliminary chitchat interaction. Our analysis reveals that our enriched dataset poses a significant challenge to these systems. Moreover, we demonstrate that our dataset can be effectively used for training purposes, enabling a system to consistently acknowledge the user's backstory while also successfully moving the task forward in the same turn, as confirmed by human evaluation. These findings highlight the benefits of generating novel chitchat-TOD scenarios to test TOD systems more thoroughly and improve their resilience to natural user interferences.

翻译：在任务型对话中，用户自然引入超出当前任务范围的闲聊，干扰对话流程。为解决此问题，无需昂贵的手动数据创建，我们利用Llama-2-70B的少样本提示增强MultiWOZ数据集，添加用户背景故事——这是任务型对话中闲聊干扰的典型示例。通过测试两个模型（一个仅基于任务型对话训练，另一个基于包含初始闲聊交互的任务型对话训练），我们评估了此添加的影响。分析表明，我们丰富后的数据集对这些系统构成了显著挑战。此外，我们证明该数据集可有效用于训练目的，使系统能够在同一轮次中一致认可用户的背景故事，同时成功推进任务，这一点通过人工评估得到确认。这些发现强调了生成新颖的闲聊-任务型对话场景对于更全面地测试任务型对话系统、提升其对自然用户干扰的适应能力具有重要价值。