Query-focused Summarization (QfS) deals with systems that generate summaries from document(s) based on a query. Motivated by the insight that Reinforcement Learning (RL) provides a generalization to Supervised Learning (SL) for Natural Language Generation, and thereby performs better (empirically) than SL, we use an RL-based approach for this task of QfS. Additionally, we also resolve the conflict of employing RL in Transformers with Teacher Forcing. We develop multiple Policy Gradient networks, trained on various reward signals: ROUGE, BLEU, and Semantic Similarity, which lead to a 10-point improvement over the State-of-the-Art approach on the ROUGE-L metric for a benchmark dataset (ELI5). We also show performance of our approach in zero-shot setting for another benchmark dataset (DebatePedia) -- our approach leads to results comparable to baselines, which were specifically trained on DebatePedia. To aid the RL training, we propose a better semantic similarity reward, enabled by a novel Passage Embedding scheme developed using Cluster Hypothesis. Lastly, we contribute a gold-standard test dataset to further research in QfS and Long-form Question Answering (LfQA).

翻译：查询聚焦摘要（QfS）处理的是根据查询从文档中生成摘要的系统。基于强化学习（RL）为自然语言生成提供了对监督学习（SL）的泛化，并且（经验上）表现优于SL这一洞见，我们采用基于RL的方法来处理QfS任务。此外，我们还解决了在Transformer中使用教师强制策略时引入RL的冲突问题。我们开发了多个策略梯度网络，这些网络基于不同的奖励信号进行训练：ROUGE、BLEU和语义相似度，在基准数据集（ELI5）上，ROUGE-L指标相比现有最优方法提升了10个点。我们还展示了我们的方法在另一个基准数据集（DebatePedia）零样本设置下的性能——我们的方法取得了与专门在DebatePedia上训练的基线方法相当的结果。为辅助RL训练，我们提出了一种更好的语义相似度奖励，该奖励基于利用聚类假设开发的新颖段落嵌入方案实现。最后，我们贡献了一个黄金标准测试数据集，以推动QfS和长句式问答（LfQA）领域的进一步研究。