In response to the pressing need for advanced clinical problem-solving tools in healthcare, we introduce BooksMed, a novel framework based on a Large Language Model (LLM). BooksMed uniquely emulates human cognitive processes to deliver evidence-based and reliable responses, utilizing the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework to effectively quantify evidence strength. For clinical decision-making to be appropriately assessed, an evaluation metric that is clinically aligned and validated is required. As a solution, we present ExpertMedQA, a multispecialty clinical benchmark comprised of open-ended, expert-level clinical questions, and validated by a diverse group of medical professionals. By demanding an in-depth understanding and critical appraisal of up-to-date clinical literature, ExpertMedQA rigorously evaluates LLM performance. BooksMed outperforms existing state-of-the-art models Med-PaLM 2, Almanac, and ChatGPT in a variety of medical scenarios. Therefore, a framework that mimics human cognitive stages could be a useful tool for providing reliable and evidence-based responses to clinical inquiries.

翻译：针对医疗领域对高级临床问题解决工具的迫切需求，我们提出BooksMed——一种基于大型语言模型（LLM）的新型框架。该框架通过模拟人类认知过程，利用GRADE（推荐意见评估、制定与评价分级）体系有效量化证据强度，从而提供基于证据的可靠回答。为合理评估临床决策，需要建立临床适配且经验证的评估指标。为此，我们提出ExpertMedQA——由开放式专家级临床问题构成的多专科临床基准，并经多领域医疗专家验证。该基准通过要求对最新临床文献进行深度理解与批判性评估，严格检验LLM性能。实验表明，BooksMed在多种医疗场景中均优于现有最优模型Med-PaLM 2、Almanac和ChatGPT。因此，模拟人类认知阶段的框架可作为可靠工具，为临床问题提供基于证据的可信回答。