The adoption of pre-trained language models in task-oriented dialogue systems has resulted in significant enhancements of their text generation abilities. However, these architectures are slow to use because of the large number of trainable parameters and can sometimes fail to generate diverse responses. To address these limitations, we propose two models with auxiliary tasks for response selection - (1) distinguishing distractors from ground truth responses and (2) distinguishing synthetic responses from ground truth labels. They achieve state-of-the-art results on the MultiWOZ 2.1 dataset with combined scores of 107.5 and 108.3 and outperform a baseline with three times more parameters. We publish reproducible code and checkpoints and discuss the effects of applying auxiliary tasks to T5-based architectures.

翻译：在任务导向对话系统中采用预训练语言模型显著提升了其文本生成能力。然而，由于大量可训练参数，这些架构运行缓慢，有时还无法生成多样化的响应。为解决这些局限性，我们提出了两种带有辅助任务用于响应选择的模型——（1）区分干扰项与真实响应；（2）区分合成响应与真实标签。它们在MultiWOZ 2.1数据集上取得了最先进的结果，综合分数分别为107.5和108.3，且性能优于参数数量多三倍的基线模型。我们发布了可复现的代码和检查点，并讨论了在基于T5的架构上应用辅助任务的效果。