Improving the performance of large language models (LLMs) in complex question-answering (QA) scenarios has always been a research focal point. Recent studies have attempted to enhance LLMs' performance by combining step-wise planning with external retrieval. While effective for advanced models like GPT-3.5, smaller LLMs face challenges in decomposing complex questions, necessitating supervised fine-tuning. Previous work has relied on manual annotation and knowledge distillation from teacher LLMs, which are time-consuming and not accurate enough. In this paper, we introduce a novel framework for enhancing LLMs' planning capabilities by using planning data derived from knowledge graphs (KGs). LLMs fine-tuned with this data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval. Evaluations on multiple datasets, including our newly proposed benchmark, highlight the effectiveness of our framework and the benefits of KG-derived planning data.

翻译：提升大语言模型在复杂问答场景中的性能一直是研究重点。近期研究尝试通过结合分步规划与外部检索来增强大语言模型的性能。虽然这对GPT-3.5等先进模型有效，但较小规模的大语言模型在分解复杂问题时面临挑战，需要进行监督微调。先前工作依赖于人工标注和从教师大语言模型进行知识蒸馏，这些方法耗时且精度不足。本文提出一种新颖框架，通过使用源自知识图谱的规划数据来增强大语言模型的规划能力。利用该数据微调的大语言模型展现出更强的规划能力，能更好地处理涉及检索的复杂问答任务。在多个数据集（包括我们新提出的基准测试）上的评估结果，凸显了我们框架的有效性以及知识图谱衍生规划数据的优势。