Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives. However, the output summary is always evaluated from summary-level which leads to the inconsistency in training and evaluation. In this paper, we propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. By modeling a contrastive objective, we show that the summarization model is able to directly generate summaries according to the summary-level score without additional modules and parameters. Extensive experiments demonstrate that COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score while preserving the parameter efficiency and inference efficiency. Compared with state-of-the-art multi-stage systems, we save more than 100 GPU training hours and obtaining 3~8 speed-up ratio during inference while maintaining comparable results.

翻译：传统的抽取式与生成式摘要系统的训练范式通常仅使用词级或句子级训练目标。然而，输出摘要始终以摘要级指标进行评估，这导致训练与评估之间存在不一致性。本文提出一种名为COLO的基于对比学习的一阶段摘要重排序框架。通过建模对比学习目标，我们证明摘要模型能够直接根据摘要级分数生成摘要，无需额外模块与参数。大量实验表明，COLO在CNN/DailyMail基准测试上将一阶段系统的抽取式与生成式结果分别提升至44.58与46.33的ROUGE-1分数，同时保持参数效率与推理效率。与当前最优的多阶段系统相比，我们在保持可比结果的同时，节省超过100 GPU训练小时，并在推理过程中获得3~8倍的加速比。