Physicians considering clinical trials for their patients are met with the laborious process of checking many text based eligibility criteria. Large Language Models (LLMs) have shown to perform well for clinical information extraction and clinical reasoning, including medical tests, but not yet in real-world scenarios. This paper investigates the use of InstructGPT to assist physicians in determining eligibility for clinical trials based on a patient's summarised medical profile. Using a prompting strategy combining one-shot, selection-inference and chain-of-thought techniques, we investigate the performance of LLMs on 10 synthetically created patient profiles. Performance is evaluated at four levels: ability to identify screenable eligibility criteria from a trial given a medical profile; ability to classify for each individual criterion whether the patient qualifies; the overall classification whether a patient is eligible for a clinical trial and the percentage of criteria to be screened by physician. We evaluated against 146 clinical trials and a total of 4,135 eligibility criteria. The LLM was able to correctly identify the screenability of 72% (2,994/4,135) of the criteria. Additionally, 72% (341/471) of the screenable criteria were evaluated correctly. The resulting trial level classification as eligible or ineligible resulted in a recall of 0.5. By leveraging LLMs with a physician-in-the-loop, a recall of 1.0 and precision of 0.71 on clinical trial level can be achieved while reducing the amount of criteria to be checked by an estimated 90%. LLMs can be used to assist physicians with pre-screening of patients for clinical trials. By forcing instruction-tuned LLMs to produce chain-of-thought responses, the reasoning can be made transparent to and the decision process becomes amenable by physicians, thereby making such a system feasible for use in real-world scenarios.

翻译：在为患者考虑临床试验时，医生需面对繁琐的基于文本的资格标准核查工作。大型语言模型（LLMs）在临床信息提取与临床推理（包括医学检验）方面表现良好，但尚未在真实场景中应用。本文探究如何利用InstructGPT模型，基于患者总结性医疗档案协助医生判断其临床试验资格。通过结合单样本学习、选择推理与思维链的提示策略，我们验证了LLMs在10个合成患者档案上的表现。评估分为四个层级：根据医疗档案识别试验中可筛选资格标准的能力；逐项判断患者是否符合标准的能力；患者是否具备临床试验资格的整体分类能力；以及需由医生核查的标准比例。我们基于146项临床试验及总计4,135条资格标准进行评估。LLM能正确识别72%（2,994/4,135）标准的可筛选性；同时，72%（341/471）的可筛选标准被正确评估。在试验层级上，患者资格分类的召回率为0.5。通过引入医生参与校验的LLM系统，临床试验层级的召回率可达1.0，精确率为0.71，同时预计可减少90%需人工核查的标准。研究表明，LLMs可用于辅助医生进行临床试验患者预筛选。通过强制指令微调后的LLM生成思维链式响应，可使其推理过程对医生透明化，决策流程更易于医生掌控，从而推动此类系统在真实场景中的应用。