Large Language Models (LLMs) are being adopted at an unprecedented rate, yet still face challenges in knowledge-intensive domains like biomedicine. Solutions such as pre-training and domain-specific fine-tuning add substantial computational overhead, requiring further domain expertise. Here, we introduce a token-optimized and robust Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework by leveraging a massive biomedical KG (SPOKE) with LLMs such as Llama-2-13b, GPT-3.5-Turbo and GPT-4, to generate meaningful biomedical text rooted in established knowledge. Compared to the existing RAG technique for Knowledge Graphs, the proposed method utilizes minimal graph schema for context extraction and uses embedding methods for context pruning. This optimization in context extraction results in more than 50% reduction in token consumption without compromising the accuracy, making a cost-effective and robust RAG implementation on proprietary LLMs. KG-RAG consistently enhanced the performance of LLMs across diverse biomedical prompts by generating responses rooted in established knowledge, accompanied by accurate provenance and statistical evidence (if available) to substantiate the claims. Further benchmarking on human curated datasets, such as biomedical true/false and multiple-choice questions (MCQ), showed a remarkable 71% boost in the performance of the Llama-2 model on the challenging MCQ dataset, demonstrating the framework's capacity to empower open-source models with fewer parameters for domain specific questions. Furthermore, KG-RAG enhanced the performance of proprietary GPT models, such as GPT-3.5 and GPT-4. In summary, the proposed framework combines explicit and implicit knowledge of KG and LLM in a token optimized fashion, thus enhancing the adaptability of general-purpose LLMs to tackle domain-specific questions in a cost-effective fashion.

翻译：大型语言模型（LLMs）正以前所未有的速度被采用，但在生物医学等知识密集型领域仍面临挑战。预训练和领域微调等解决方案会额外增加巨大的计算开销，并需要进一步的领域专业知识。在此，我们提出一种基于知识图谱的检索增强生成（KG-RAG）框架，该框架采用令牌优化且稳健的设计，通过利用包含海量生物医学知识图谱（SPOKE）的LLMs（如Llama-2-13b、GPT-3.5-Turbo和GPT-4），生成根植于已有知识的具有意义的生物医学文本。与现有针对知识图谱的RAG技术相比，所提方法利用最小图模式进行上下文提取，并采用嵌入方法进行上下文剪枝。这种上下文提取优化使得令牌消耗减少超过50%，同时不牺牲准确性，从而在专有LLMs上实现了高成本效益且稳健的RAG实现。KG-RAG通过生成基于已有知识的响应，并附有准确的来源和统计证据（如可用）来支撑论断，持续提升了各类生物医学提示下LLMs的性能。在人工标注数据集（如生物医学判断题和选择题（MCQ））上的进一步基准测试显示，Llama-2模型在具有挑战性的MCQ数据集上性能提升高达71%，证明了该框架能够赋能参数较少的开源模型应对领域特定问题。此外，KG-RAG还提升了GPT-3.5和GPT-4等专有GPT模型的性能。综上所述，所提框架以令牌优化的方式结合了知识图谱的显式知识与LLMs的隐式知识，从而以一种高成本效益的方式增强了通用LLMs应对领域特定问题的适应性。