Query-focused summarization (QFS) aims to provide a summary of a document that satisfies information need of a given query and is useful in various IR applications, such as abstractive snippet generation. Current QFS approaches typically involve injecting additional information, e.g. query-answer relevance or fine-grained token-level interaction between a query and document, into a finetuned large language model. However, these approaches often require extra parameters \& training, and generalize poorly to new dataset distributions. To mitigate this, we propose leveraging a recently developed constrained generation model Neurological Decoding (NLD) as an alternative to current QFS regimes which rely on additional sub-architectures and training. We first construct lexical constraints by identifying important tokens from the document using a lightweight gradient attribution model, then subsequently force the generated summary to satisfy these constraints by directly manipulating the final vocabulary likelihood. This lightweight approach requires no additional parameters or finetuning as it utilizes both an off-the-shelf neural retrieval model to construct the constraints and a standard generative language model to produce the QFS. We demonstrate the efficacy of this approach on two public QFS collections achieving near parity with the state-of-the-art model with substantially reduced complexity.

翻译：查询聚焦摘要旨在提供满足给定查询信息需求的文档摘要，在抽象片段生成等多项信息检索应用中具有实用价值。当前方法通常需将额外信息（如查询-答案相关性或查询与文档间的细粒度标记级交互）注入微调后的大语言模型。然而这些方法常需额外参数与训练，且难以泛化至新数据集分布。为此，我们提出利用近期开发的约束生成模型Neurological Decoding作为替代方案，摒弃依赖附加子架构与训练的现有方法。我们首先通过轻量级梯度归因模型从文档中识别关键标记构建词汇约束，继而通过直接操纵最终词汇概率强制生成摘要满足这些约束。该轻量方法无需额外参数或微调，因其同时利用现成神经检索模型构建约束和标准生成型语言模型进行查询聚焦摘要。我们在两个公开查询聚焦摘要数据集上验证了方法的有效性，以显著降低的复杂度实现了与最先进模型近乎相当的性能。