In the face of rapidly expanding online medical literature, automated systems for aggregating and summarizing information are becoming increasingly crucial for healthcare professionals and patients. Large Language Models (LLMs), with their advanced generative capabilities, have shown promise in various NLP tasks, and their potential in the healthcare domain, particularly for Closed-Book Generative QnA, is significant. However, the performance of these models in domain-specific tasks such as medical Q&A remains largely unexplored. This study aims to fill this gap by comparing the performance of general and medical-specific distilled LMs for medical Q&A. We aim to evaluate the effectiveness of fine-tuning domain-specific LMs and compare the performance of different families of Language Models. The study will address critical questions about these models' reliability, comparative performance, and effectiveness in the context of medical Q&A. The findings will provide valuable insights into the suitability of different LMs for specific applications in the medical domain.

翻译：随着在线医学文献的快速扩展，为医疗专业人员及患者聚合与总结信息的自动化系统正变得愈发关键。大语言模型凭借其先进的生成能力，已在各类自然语言处理任务中展现出潜力，尤其在医疗领域（特别是闭卷生成式问答）的应用前景显著。然而，这些模型在医学问答等特定领域任务中的表现仍鲜有研究。本研究旨在通过对比通用型与医学专用蒸馏语言模型在医学问答中的表现来填补这一空白。我们评估了微调领域专用语言模型的有效性，并比较了不同系列语言模型的性能。研究将围绕医学问答场景中这些模型的可靠性、相对性能及有效性等关键问题展开探讨。研究结果将为不同语言模型在医学领域特定应用中的适用性提供重要见解。