A robust summarization system should be able to capture the gist of the document, regardless of the specific word choices or noise in the input. In this work, we first explore the summarization models' robustness against perturbations including word-level synonym substitution and noise. To create semantic-consistent substitutes, we propose a SummAttacker, which is an efficient approach to generating adversarial samples based on language models. Experimental results show that state-of-the-art summarization models have a significant decrease in performance on adversarial and noisy test sets. Next, we analyze the vulnerability of the summarization systems and explore improving the robustness by data augmentation. Specifically, the first brittleness factor we found is the poor understanding of infrequent words in the input. Correspondingly, we feed the encoder with more diverse cases created by SummAttacker in the input space. The other factor is in the latent space, where the attacked inputs bring more variations to the hidden states. Hence, we construct adversarial decoder input and devise manifold softmixing operation in hidden space to introduce more diversity. Experimental results on Gigaword and CNN/DM datasets demonstrate that our approach achieves significant improvements over strong baselines and exhibits higher robustness on noisy, attacked, and clean datasets.

翻译：一个鲁棒的摘要系统应能够捕捉文档的核心内容，不受输入中特定词语选择或噪声的影响。本文首先探究了摘要模型对词级同义词替换和噪声等扰动的鲁棒性。为生成语义一致的替换词，我们提出了一种基于语言模型的高效对抗样本生成方法——SummAttacker。实验结果表明，当前最先进的摘要模型在对抗性和含噪声的测试集上性能显著下降。随后，我们分析了摘要系统的脆弱性，并探索通过数据增强提升鲁棒性。具体而言，我们发现的第一类脆弱因素是对输入中低频词的理解不足。相应地，我们通过SummAttacker在输入空间中生成更多样化的案例来训练编码器。另一类因素存在于隐空间：被攻击的输入会导致隐藏状态产生更多变化。因此，我们构建了对抗性解码器输入，并在隐空间中设计流形软混合操作以引入更多多样性。在Gigaword和CNN/DM数据集上的实验结果表明，我们的方法相较于强基线取得了显著提升，且在含噪声、受攻击和清洁数据集上均展现出更高的鲁棒性。