The large language model called ChatGPT has drawn extensively attention because of its human-like expression and reasoning abilities. In this study, we investigate the feasibility of using ChatGPT in experiments on using ChatGPT to translate radiology reports into plain language for patients and healthcare providers so that they are educated for improved healthcare. Radiology reports from 62 low-dose chest CT lung cancer screening scans and 76 brain MRI metastases screening scans were collected in the first half of February for this study. According to the evaluation by radiologists, ChatGPT can successfully translate radiology reports into plain language with an average score of 4.27 in the five-point system with 0.08 places of information missing and 0.07 places of misinformation. In terms of the suggestions provided by ChatGPT, they are general relevant such as keeping following-up with doctors and closely monitoring any symptoms, and for about 37% of 138 cases in total ChatGPT offers specific suggestions based on findings in the report. ChatGPT also presents some randomness in its responses with occasionally over-simplified or neglected information, which can be mitigated using a more detailed prompt. Furthermore, ChatGPT results are compared with a newly released large model GPT-4, showing that GPT-4 can significantly improve the quality of translated reports. Our results show that it is feasible to utilize large language models in clinical education, and further efforts are needed to address limitations and maximize their potential.
翻译:大型语言模型ChatGPT因其类人表达与推理能力而受到广泛关注。本研究探讨了在实验中使用ChatGPT将放射学报告翻译成通俗语言的可能性,以便患者和医疗提供者能够通过教育改善医疗保健。研究收集了2023年2月上半月的62份低剂量胸部CT肺癌筛查报告和76份脑MRI转移瘤筛查报告。根据放射科医生的评估,ChatGPT成功将放射学报告翻译成通俗语言,在5分制中平均得分为4.27,信息缺失点数为0.08,错误信息点数为0.07。ChatGPT提供的建议普遍具有相关性,例如建议随访医生和密切监测症状;在总共138例病例中,约37%的案例中ChatGPT根据报告发现给出了具体建议。ChatGPT的回复也呈现出一定随机性,偶尔出现过度简化或遗漏信息的情况,可通过更详细的提示词加以缓解。此外,将ChatGPT结果与新发布的大型模型GPT-4进行比较,显示GPT-4能显著提高翻译报告质量。我们的结果表明,大型语言模型在临床教育中具有可行性,但需进一步努力解决其局限性并发挥其潜力。