Task-oriented dialogue systems (TODS) have become crucial for users to interact with machines and computers using natural language. One of its key components is the dialogue manager, which guides the conversation towards a good goal for the user by providing the best possible response. Previous works have proposed rule-based systems (RBS), reinforcement learning (RL), and supervised learning (SL) as solutions for the correct dialogue management; in other words, select the best response given input by the user. However, this work argues that the leading cause of DMs not achieving maximum performance resides in the quality of the datasets rather than the models employed thus far; this means that dataset errors, like mislabeling, originate a large percentage of failures in dialogue management. We studied the main errors in the most widely used datasets, Multiwoz 2.1 and SGD, to demonstrate this hypothesis. To do this, we have designed a synthetic dialogue generator to fully control the amount and type of errors introduced in the dataset. Using this generator, we demonstrated that errors in the datasets contribute proportionally to the performance of the models

翻译：任务型对话系统（TODS）已成为用户通过自然语言与机器及计算机交互的重要工具。其中关键组件之一是对话管理器，它通过提供最优响应引导对话走向用户期望的目标。现有研究已提出基于规则的对话系统（RBS）、强化学习（RL）和监督学习（SL）作为解决正确对话管理问题的方案，即根据用户输入选择最佳响应。然而，本文认为导致对话管理器（DM）性能未达最优的主要原因在于数据集质量，而非当前采用模型的局限性——这意味着数据集错误（如标签错配）会引发对话管理中的大部分故障。为验证该假设，我们分析了最广泛使用的数据集Multiwoz 2.1和SGD中的主要错误类型。为此，我们设计了一个合成对话生成器，实现对数据集中错误数量与类型的完全可控。通过该生成器，我们证明了数据集中的错误会按比例影响模型性能。