In this work, we take a first step towards designing summarization systems that are faithful to the author's opinions and perspectives. Focusing on a case study of preserving political perspectives in news summarization, we find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries, misrepresenting the intent and perspectives of the news authors. We thus propose P^3Sum, a diffusion model-based summarization approach controlled by political perspective classifiers. In P^3Sum, the political leaning of a generated summary is iteratively evaluated at each decoding step, and any drift from the article's original stance incurs a loss back-propagated to the embedding layers, steering the political stance of the summary at inference time. Extensive experiments on three news summarization datasets demonstrate that P^3Sum outperforms state-of-the-art summarization systems and large language models by up to 11.4% in terms of the success rate of stance preservation, with on-par performance on standard summarization utility metrics. These findings highlight the lacunae that even for state-of-the-art models it is still challenging to preserve author perspectives in news summarization, while P^3Sum presents an important first step towards evaluating and developing summarization systems that are faithful to author intent and perspectives.
翻译:在本工作中,我们迈出了设计忠实于作者观点与立场的摘要系统的第一步。聚焦于新闻摘要中保留政治视角的案例研究,我们发现现有方法在超过50%的摘要中改变了新闻文章的政治观点与立场,歪曲了新闻作者的意图和视角。为此,我们提出P^3Sum——一种由政治立场分类器控制的扩散模型摘要方法。在P^3Sum中,生成摘要的政治倾向在每个解码步骤中被迭代评估,一旦偏离原文立场,偏差损失将反向传播至嵌入层,从而在推理阶段引导摘要的政治立场。在三个新闻摘要数据集上的大量实验表明,P^3Sum在立场保留成功率上以高达11.4%的优势超越现有最优摘要系统及大语言模型,同时在标准摘要效用指标上表现相当。这些发现揭示了即使是最先进模型在新闻摘要中保留作者视角仍面临挑战的现状,而P^3Sum则朝着评估与开发忠实于作者意图与视角的摘要系统迈出了关键的第一步。