Recent advances in large language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, their effective application in the medical domain is hampered by a lack of medical domain knowledge. In this study, we present SA-MDKIF, a scalable and adaptable framework that aims to inject medical knowledge into general-purpose LLMs through instruction tuning, thereby enabling adaptability for various downstream tasks. SA-MDKIF consists of two stages: skill training and skill adaptation. In the first stage, we define 12 basic medical skills and use AdaLoRA to train these skills based on uniformly formatted instructional datasets that we have constructed. In the next stage, we train the skill router using task-specific downstream data and use this router to integrate the acquired skills with LLMs during inference. Experimental results on 9 different medical tasks show that SA-MDKIF improves performance by 10-20% compared to the original LLMs. Notably, this improvement is particularly pronounced for unseen medical tasks, showing an improvement of up to 30%.

翻译：摘要：大型语言模型（LLM）在各类自然语言处理（NLP）任务中展现出卓越性能，但其在医学领域的有效应用受到缺乏医学领域知识的制约。本研究提出SA-MDKIF——一种可扩展且自适应的框架，旨在通过指令微调将医学知识注入通用LLM，从而实现对多种下游任务的适应性。SA-MDKIF包含两个阶段：技能训练与技能适配。第一阶段，我们定义了12项基础医学技能，并基于自建的统一格式指令数据集，利用AdaLoRA训练这些技能。第二阶段，我们使用任务特定下游数据训练技能路由器，并在推理阶段通过该路由器将习得技能与LLM整合。在9项不同医学任务上的实验结果表明，与原始LLM相比，SA-MDKIF的性能提升10%-20%。值得注意的是，对于未见医学任务，性能提升尤为显著，最高可达30%。