We introduce inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models, imitating human summarization behaviors. Our IRL model estimates the reward function using a suite of important sub-rewards for summarization and concurrently optimizes the policy network. Experimental results across datasets in different domains (CNN/DailyMail and WikiHow) and various model sizes (BART-base and BART-large) demonstrate the superiority of our proposed IRL model for summarization over MLE and RL baselines. The resulting summaries exhibit greater similarity to human-crafted gold references, outperforming MLE and RL baselines on metrics such as ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations.

翻译：我们提出将逆向强化学习（IRL）作为一种有效的范式，用于训练生成式摘要模型以模仿人类摘要行为。我们的IRL模型通过一组用于摘要任务的重要子奖励来估计奖励函数，并同步优化策略网络。在跨不同领域（CNN/DailyMail与WikiHow）的数据集以及多种模型规模（BART-base和BART-large）上的实验结果表明，我们提出的IRL摘要模型在性能上优于MLE和RL基线方法。生成的摘要与人工撰写的黄金参考摘要具有更高相似度，在ROUGE、覆盖度、新颖性、压缩比、事实一致性及人工评估等指标上均超越MLE和RL基线方法。