Multi-party dialogues are more difficult for models to understand than one-to-one two-party dialogues, since they involve multiple interlocutors, resulting in interweaving reply-to relations and information flows. To step over these obstacles, an effective way is to pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying. However, due to the lack of explicitly annotated discourse labels in multi-party dialogue corpora, previous works fail to scale up the pre-training process by putting aside the unlabeled multi-party conversational data for nothing. To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model by unsupervised latent variable inference methods. Experiments on multiple downstream tasks show that our pre-trained model outperforms strong baselines by large margins and achieves state-of-the-art (SOTA) results, justifying the effectiveness of our method. The official implementation of this paper is available at https://github.com/EricLee8/MPD_EMVI.

翻译：多方对话比一对一的双人对话更难让模型理解，因为其涉及多个对话者，导致回复关系和信息流相互交织。为克服这些障碍，一种有效方法是预训练一个能理解多方对话话语结构（即每条话语回复对象）的模型。然而，由于多方对话语料库缺乏显式标注的话语标签，以往研究未能充分利用未标注的多方对话数据进行大规模预训练。为充分利用未标注数据，我们提出将话语结构视为隐变量，通过无监督隐变量推理方法联合推断该结构并预训练话语感知模型。多项下游任务实验表明，我们的预训练模型大幅超越强基线方法并取得最先进结果，验证了该方法的有效性。本文官方代码实现见 https://github.com/EricLee8/MPD_EMVI。