Task-oriented dialogue systems are broadly used in virtual assistants and other automated services, providing interfaces between users and machines to facilitate specific tasks. Nowadays, task-oriented dialogue systems have greatly benefited from pre-trained language models (PLMs). However, their task-solving performance is constrained by the inherent capacities of PLMs, and scaling these models is expensive and complex as the model size becomes larger. To address these challenges, we propose Soft Mixture-of-Expert Task-Oriented Dialogue system (SMETOD) which leverages an ensemble of Mixture-of-Experts (MoEs) to excel at subproblems and generate specialized outputs for task-oriented dialogues. SMETOD also scales up a task-oriented dialogue system with simplicity and flexibility while maintaining inference efficiency. We extensively evaluate our model on three benchmark functionalities: intent prediction, dialogue state tracking, and dialogue response generation. Experimental results demonstrate that SMETOD achieves state-of-the-art performance on most evaluated metrics. Moreover, comparisons against existing strong baselines show that SMETOD has a great advantage in the cost of inference and correctness in problem-solving.

翻译：任务导向对话系统广泛应用于虚拟助手及其他自动化服务中，通过提供用户与机器之间的交互接口来促进特定任务的完成。当前，任务导向对话系统已大幅受益于预训练语言模型。然而，其任务解决能力受限于预训练语言模型的固有容量，且随着模型规模增大，扩展这些模型的成本高昂且过程复杂。为应对这些挑战，我们提出了软性混合专家任务导向对话系统，该系统利用混合专家模型集成来擅长处理子问题，并为任务导向对话生成专业化输出。SMETOD在保持推理效率的同时，能以简单灵活的方式扩展任务导向对话系统。我们在三个基准功能上进行了广泛评估：意图预测、对话状态跟踪和对话响应生成。实验结果表明，SMETOD在大多数评估指标上达到了最优性能。此外，与现有强基线模型的对比表明，SMETOD在推理成本与问题解决正确性方面具有显著优势。