Existing dialogue modeling methods have achieved promising performance on various dialogue tasks with the aid of Transformer and the large-scale pre-trained language models. However, some recent studies revealed that the context representations produced by these methods suffer the problem of anisotropy. In this paper, we find that the generated representations are also not conversational, losing the conversation structure information during the context modeling stage. To this end, we identify two properties in dialogue modeling, i.e., locality and isotropy, and present a simple method for dialogue representation calibration, namely SimDRC, to build isotropic and conversational feature spaces. Experimental results show that our approach significantly outperforms the current state-of-the-art models on three dialogue tasks across the automatic and human evaluation metrics. More in-depth analyses further confirm the effectiveness of our proposed approach.

翻译：现有对话建模方法借助Transformer和大规模预训练语言模型，已在各类对话任务上取得显著成效。然而，近期研究表明这些方法产生的上下文表示存在各向异性问题。本文发现生成的表示同样缺乏对话性，在上下文建模阶段丢失了对话结构信息。为此，我们识别出对话建模中的两个特性，即局部性和各向同性，并提出一种简洁的对话表示校准方法SimDRC，以构建各向同性且具有对话性的特征空间。实验结果表明，在基于自动评估和人工评估指标的三项对话任务中，我们的方法显著优于当前最先进模型。更深入的分析进一步证实了所提方法的有效性。