Large language models (LLMs) demonstrate remarkable medical expertise, but data privacy concerns impede their direct use in healthcare environments. Although offering improved data privacy protection, domain-specific small language models (SLMs) often underperform LLMs, emphasizing the need for methods that reduce this performance gap while alleviating privacy concerns. In this paper, we present a simple yet effective method that harnesses LLMs' medical proficiency to boost SLM performance in medical tasks under privacy-restricted scenarios. Specifically, we mitigate patient privacy issues by extracting keywords from medical data and prompting the LLM to generate a medical knowledge-intensive context by simulating clinicians' thought processes. This context serves as additional input for SLMs, augmenting their decision-making capabilities. Our method significantly enhances performance in both few-shot and full training settings across three medical knowledge-intensive tasks, achieving up to a 22.57% increase in absolute accuracy compared to SLM fine-tuning without context, and sets new state-of-the-art results in two medical tasks within privacy-restricted scenarios. Further out-of-domain testing and experiments in two general domain datasets showcase its generalizability and broad applicability. Our code can be found at https://github.com/XZhang97666/PrivacyBoost-SLM.

翻译：大型语言模型（LLMs）展现出卓越的医学专业知识，但数据隐私问题阻碍其在医疗环境中的直接应用。尽管领域专用小型语言模型（SLMs）能提供更好的数据隐私保护，但其性能往往不及LLMs，亟需开发既能缩小性能差距又能缓解隐私问题的方法。本文提出一种简单有效的方法，在隐私受限场景下利用LLMs的医学专业能力提升SLMs的医疗任务表现。具体而言，我们通过从医学数据中提取关键词，并提示LLM模拟临床医生的思维过程生成富含医学知识的情境，从而缓解患者隐私问题。该情境作为SLMs的额外输入，增强其决策能力。在三个医学知识密集型任务中，本方法在少样本和全量训练设置下均显著提升性能，与未引入情境的SLM微调相比，绝对准确率最高提升22.57%，并在隐私受限场景下的两个医学任务中创下最新最优结果。跨领域测试及两个通用领域数据集的实验进一步证明了其泛化性和广泛适用性。我们的代码已开源至https://github.com/XZhang97666/PrivacyBoost-SLM。