Learning from free-text human feedback is essential for dialog systems, but annotated data is scarce and usually covers only a small fraction of error types known in conversational AI. Instead of collecting and annotating new datasets from scratch, recent advances in synthetic dialog generation could be used to augment existing dialog datasets with the necessary annotations. However, to assess the feasibility of such an effort, it is important to know the types and frequency of free-text human feedback included in these datasets. In this work, we investigate this question for a variety of commonly used dialog datasets, including MultiWoZ, SGD, BABI, PersonaChat, Wizards-of-Wikipedia, and the human-bot split of the Self-Feeding Chatbot. Using our observations, we derive new taxonomies for the annotation of free-text human feedback in dialogs and investigate the impact of including such data in response generation for three SOTA language generation models, including GPT-2, LLAMA, and Flan-T5. Our findings provide new insights into the composition of the datasets examined, including error types, user response types, and the relations between them.

翻译：从自由文本人类反馈中学习对于对话系统至关重要，但带注释的数据稀缺，通常仅覆盖对话式AI中已知错误类型的一小部分。与其从头开始收集和注释新数据集，近期合成对话生成的进展可用于利用必要的注释扩充现有对话数据集。然而，要评估这种努力的可行性，了解这些数据集中包含的自由文本人类反馈的类型和频率至关重要。在本研究中，我们针对多种常用对话数据集（包括MultiWoZ、SGD、BABI、PersonaChat、Wizards-of-Wikipedia以及Self-Feeding Chatbot的人机分离部分）探讨了这一问题。基于我们的观察，我们推导出用于对话中自由文本人类反馈注释的新分类法，并研究了在三种最先进的语言生成模型（包括GPT-2、LLAMA和Flan-T5）的响应生成中包含此类数据的影响。我们的发现为所检查数据集的构成提供了新的见解，包括错误类型、用户响应类型以及它们之间的关系。