Dialogue summarization aims to condense the lengthy dialogue into a concise summary, and has recently achieved significant progress. However, the result of existing methods is still far from satisfactory. Previous works indicated that omission is a major factor in affecting the quality of summarization, but few of them have further explored the omission problem, such as how omission affects summarization results and how to detect omission, which is critical for reducing omission and improving summarization quality. Moreover, analyzing and detecting omission relies on summarization datasets with omission labels (i.e., which dialogue utterances are omitted in the summarization), which are not available in the current literature. In this paper, we propose the OLDS dataset, which provides high-quality Omission Labels for Dialogue Summarization. By analyzing this dataset, we find that a large improvement in summarization quality can be achieved by providing ground-truth omission labels for the summarization model to recover omission information, which demonstrates the importance of omission detection for omission mitigation in dialogue summarization. Therefore, we formulate an omission detection task and demonstrate our proposed dataset can support the training and evaluation of this task well. We also call for research action on omission detection based on our proposed datasets. Our dataset and codes are publicly available.
翻译:对话摘要旨在将冗长对话压缩成简洁的摘要,近年来取得了显著进展。然而,现有方法的结果仍远未令人满意。先前研究表明,省略是影响摘要质量的主要因素,但很少有研究进一步探讨省略问题,例如省略如何影响摘要结果以及如何检测省略——这些对减少省略、提升摘要质量至关重要。此外,分析和检测省略依赖于带有省略标签的摘要数据集(即哪些对话语句在摘要中被省略),而当前文献中尚无此类数据集。本文提出了OLDS数据集,为对话摘要提供了高质量的省略标签。通过分析该数据集,我们发现,若向摘要模型提供真实的省略标签以恢复省略信息,摘要质量可获显著提升,这证明了省略检测对于减轻对话摘要中省略问题的重要性。因此,我们定义了省略检测任务,并表明所提出的数据集能有效支持该任务的训练与评估。我们也呼吁基于所提数据集开展省略检测研究。我们的数据集与代码已公开。