Recent studies have investigated utilizing Knowledge Graphs (KGs) to enhance Quesetion Answering (QA) performance of Large Language Models (LLMs), yet structured KG verbalization remains challengin. Existing methods, such as triple-form or free-form textual conversion of triple-form facts, encounter several issues. These include reduced evidence density due to duplicated entities or relationships, and reduced evidence clarity due to an inability to emphasize crucial evidence. To address these issues, we propose EFSum, an Evidence-focused Fact Summarization framework for enhanced QA with knowledge-augmented LLMs. We optimize an open-source LLM as a fact summarizer through distillation and preference alignment. Our extensive experiments show that EFSum improves LLM's zero-shot QA performance, and it is possible to ensure both the helpfulness and faithfulness of the summary.

翻译：近期研究探索了利用知识图谱（KG）提升大语言模型（LLM）的问答（QA）性能，但结构化知识图谱的文本化处理仍具挑战性。现有方法（如三元组形式或三元组事实的自由形式文本转换）存在若干问题：重复实体或关系导致证据密度降低，且无法突出关键证据致使证据清晰度下降。针对这些问题，我们提出EFSum——面向证据的事实摘要框架，用于增强知识增强型大语言模型的问答能力。我们通过蒸馏和偏好对齐技术，将开源大语言模型优化为事实摘要生成器。大量实验表明，EFSum能有效提升大语言模型的零样本问答性能，同时确保摘要兼具实用性与忠实性。