Pre-trained conversation models (PCMs) have demonstrated remarkable results in task-oriented dialogue (TOD) systems. Many PCMs focus predominantly on dialogue management tasks like dialogue state tracking, dialogue generation tasks like response generation, or both. However, the existing PCMs seldom consider dialogue comprehension tasks, such as dialogue question answering and summarization tasks. These tasks allow PCMs to glean dialogue context from various angles. This observation naturally raises the question: Can the performance of downstream dialogue tasks be enhanced if a PCM is pre-trained on dialogue management, generation, and comprehension tasks? To investigate this, we proposed an Omnipotent Dialogue pre-training model (OmniDialog). It unifies these three dialogue tasks into a monolithic framework by multi-task learning, fostering inter-task communication. The pre-training corpus of OmniDialog spans $\mathbf{7}$ dialogue-focused tasks, drawing from $\mathbf{15}$ datasets and encompassing over $\mathbf{3.2}$ million dialogue utterances. To our knowledge, OmniDialog is a pioneering PCM pre-trained across dialogue management, generation, and comprehension domains. We evaluated its performance across four tasks: dialogue summarization, end-to-end dialogue modeling, dialogue state tracking, and intent classification. The results underscore its efficacy in domain transfer learning, low-resource, and full-dataset scenarios. Furthermore, to glean a nuanced understanding of OmniDialog's strengths and potential pitfalls, we designed a fine-grained analysis framework for dialogue-centric tasks. Experimental results show that the OmniDialog is good at hard samples, such as long dialogues and lengthy responses.

翻译：预训练对话模型（PCMs）在任务型对话（TOD）系统中已展现出显著成效。多数PCMs主要聚焦于对话管理任务（如对话状态跟踪）和对话生成任务（如回复生成），或两者兼而有之。然而，现有PCMs鲜少考虑对话理解任务，例如对话问答和摘要任务。这些任务使PCMs能够从多角度提取对话语境。这一观察自然引发疑问：若PCM在对话管理、生成与理解任务上均进行预训练，能否提升下游对话任务的性能？为探究此问题，我们提出了一种全能型对话预训练模型（OmniDialog）。该模型通过多任务学习将三类对话任务统一至单一框架内，促进任务间交互。OmniDialog的预训练语料涵盖$\mathbf{7}$项以对话为核心的任务，源自$\mathbf{15}$个数据集，包含超过$\mathbf{3.2}$百万条对话语句。据我们所知，OmniDialog是首个跨对话管理、生成与理解领域进行预训练的PCM。我们在对话摘要、端到端对话建模、对话状态跟踪及意图分类四项任务上评估其性能。结果表明，该模型在领域迁移学习、低资源设定及全数据集场景下均表现出色。此外，为深入理解OmniDialog的优势与潜在局限，我们设计了面向对话任务的细粒度分析框架。实验显示，OmniDialog在处理困难样本（如长对话与长回复）方面表现优异。