This paper summarizes our contributions to the document-grounded dialog tasks at the 9th and 10th Dialog System Technology Challenges (DSTC9 and DSTC10). In both iterations the task consists of three subtasks: first detect whether the current turn is knowledge seeking, second select a relevant knowledge document, and third generate a response grounded on the selected document. For DSTC9 we proposed different approaches to make the selection task more efficient. The best method, Hierarchical Selection, actually improves the results compared to the original baseline and gives a speedup of 24x. In the DSTC10 iteration of the task, the challenge was to adapt systems trained on written dialogs to perform well on noisy automatic speech recognition transcripts. Therefore, we proposed data augmentation techniques to increase the robustness of the models as well as methods to adapt the style of generated responses to fit well into the proceeding dialog. Additionally, we proposed a noisy channel model that allows for increasing the factuality of the generated responses. In addition to summarizing our previous contributions, in this work, we also report on a few small improvements and reconsider the automatic evaluation metrics for the generation task which have shown a low correlation to human judgments.
翻译:本文总结了我们在第九届和第十届对话系统技术挑战赛(DSTC9和DSTC10)的文档基础对话任务中的贡献。在这两届比赛中,任务包含三个子任务:首先检测当前轮次是否在寻求知识,其次选择相关的知识文档,最后基于所选文档生成回应。针对DSTC9,我们提出了多种方法以提高选择任务的效率。最佳方法——层次化选择(Hierarchical Selection)——不仅相比原始基线提升了结果,还实现了24倍的加速。在DSTC10的任务迭代中,挑战在于使基于书面对话训练的系统能良好处理带噪的自动语音识别转录文本。为此,我们提出了数据增强技术以提升模型的鲁棒性,以及调整生成回应风格以使其与后续对话自然衔接的方法。此外,我们提出了一种噪声信道模型,用于提高生成回应的真实性。除总结先前贡献外,本文还报告了一些小型改进,并重新审视了生成任务的自动评估指标——这些指标与人工判断的相关性较低。