Discourse processing suffers from data sparsity, especially for dialogues. As a result, we explore approaches to build discourse structures for dialogues, based on attention matrices from Pre-trained Language Models (PLMs). We investigate multiple tasks for fine-tuning and show that the dialogue-tailored Sentence Ordering task performs best. To locate and exploit discourse information in PLMs, we propose an unsupervised and a semi-supervised method. Our proposals achieve encouraging results on the STAC corpus, with F1 scores of 57.2 and 59.3 for unsupervised and semi-supervised methods, respectively. When restricted to projective trees, our scores improved to 63.3 and 68.1.

翻译：话语处理面临数据稀疏性问题，尤其是在对话场景中。为此，我们探索了基于预训练语言模型注意力矩阵构建对话话语结构的方法。我们研究了多种微调任务，并证明面向对话的句子排序任务表现最佳。为定位并利用预训练语言模型中的话语信息，我们提出了无监督和半监督两种方法。在STAC语料库上，我们的方法取得了令人鼓舞的结果：无监督方法的F1得分为57.2，半监督方法为59.3。当限定投影树结构时，得分分别提升至63.3和68.1。