Large language models (LLMs) can generate fluent summaries across domains using prompting techniques, reducing the need to train models for summarization applications. However, crafting effective prompts that guide LLMs to generate summaries with the appropriate level of detail and writing style remains a challenge. In this paper, we explore the use of salient information extracted from the source document to enhance summarization prompts. We show that adding keyphrases in prompts can improve ROUGE F1 and recall, making the generated summaries more similar to the reference and more complete. The number of keyphrases can control the precision-recall trade-off. Furthermore, our analysis reveals that incorporating phrase-level salient information is superior to word- or sentence-level. However, the impact on hallucination is not universally positive across LLMs. To conduct this analysis, we introduce Keyphrase Signal Extractor (SigExt), a lightweight model that can be finetuned to extract salient keyphrases. By using SigExt, we achieve consistent ROUGE improvements across datasets and open-weight and proprietary LLMs without any LLM customization. Our findings provide insights into leveraging salient information in building prompt-based summarization systems. We release our code at \url{https://github.com/amazon-science/SigExt}

翻译：大型语言模型（LLMs）能够通过提示技术生成跨领域的流畅摘要，从而减少为摘要应用训练模型的需求。然而，设计有效的提示以引导LLMs生成具有适当细节水平和写作风格的摘要仍具挑战性。本文探索利用从源文档中提取的显著信息来增强摘要提示。研究表明，在提示中添加关键短语能够提升ROUGE F1值和召回率，使生成的摘要与参考摘要更相似且更完整。关键短语的数量可以控制精度与召回率的权衡。进一步分析表明，融入短语级显著信息优于词级或句子级信息。但不同LLMs在幻觉抑制方面的影响并不一致。为开展此项分析，我们提出了关键短语信号提取器（SigExt）——一种可通过微调提取显著关键短语的轻量级模型。使用SigExt可在无需任何LLM定制的情况下，在不同数据集及开源与专有LLMs上实现一致的ROUGE指标提升。本研究为构建基于提示的摘要系统中利用显著信息提供了实践洞见。代码已发布于\url{https://github.com/amazon-science/SigExt}