We investigate response generation for multi-turn dialogue in generative-based chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the sequences, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) component without posterior knowledge through introducing a recurrent summarizing variable into the GRU, which can aggregate the accumulated distribution variations of subsequences. PVGRU can perceive the subtle semantic variability through summarizing variables that are optimized by the devised distribution consistency and reconstruction objectives. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.

翻译：我们研究基于生成式聊天机器人的多轮对话中的回复生成。现有基于RNN（循环神经网络）的生成模型通常使用最后的隐藏状态来概括序列，这使得模型无法捕捉不同对话中观察到的细微变化，也无法区分结构相似对话之间的差异。本文提出一种无需后验知识的伪变分门控循环单元（PVGRU）组件，通过将循环总结变量引入GRU，该组件能够聚合子序列的累积分布变化。PVGRU可通过由设计的分布一致性和重构目标优化的总结变量来感知细微的语义变化。此外，我们基于PVGRU构建了伪变分分层对话（PVHD）模型。实验结果表明，PVGRU能够在两个基准数据集上显著提升回复的多样性和相关性。