Artificial intelligence is gaining traction in more ways than ever before. The popularity of language models and AI-based businesses has soared since ChatGPT was made available to the general public via OpenAI. It is becoming increasingly common for people to use ChatGPT both professionally and personally. Considering the widespread use of ChatGPT and the reliance people place on it, this study determined how reliable ChatGPT can be for answering complex medical and clinical questions. Harvard University gross anatomy along with the United States Medical Licensing Examination (USMLE) questionnaire were used to accomplish the objective. The paper evaluated the obtained results using a 2-way ANOVA and posthoc analysis. Both showed systematic covariation between format and prompt. Furthermore, the physician adjudicators independently rated the outcome's accuracy, concordance, and insight. As a result of the analysis, ChatGPT-generated answers were found to be more context-oriented and represented a better model for deductive reasoning than regular Google search results. Furthermore, ChatGPT obtained 58.8% on logical questions and 60% on ethical questions. This means that the ChatGPT is approaching the passing range for logical questions and has crossed the threshold for ethical questions. The paper believes ChatGPT and other language learning models can be invaluable tools for e-learners; however, the study suggests that there is still room to improve their accuracy. In order to improve ChatGPT's performance in the future, further research is needed to better understand how it can answer different types of questions.
翻译:人工智能正以前所未有的方式获得发展势头。自OpenAI向公众开放ChatGPT以来,语言模型和基于人工智能的业务的受欢迎程度急剧上升。人们在专业和个人领域使用ChatGPT的情况越来越普遍。鉴于ChatGPT的广泛使用以及人们对它的依赖,本研究确定了ChatGPT在回答复杂的医学和临床问题方面的可靠性。为了实现这一目标,使用了哈佛大学大体解剖学以及美国医学执照考试(USMLE)的问卷。本文采用双因素方差分析和事后分析对所得结果进行了评估。两者均表明格式与提示之间存在系统性的协变关系。此外,医师评审员独立评估了结果的准确性、一致性和洞察力。分析结果表明,与常规谷歌搜索结果相比,ChatGPT生成的答案更具情境导向,代表了更好的演绎推理模型。此外,ChatGPT在逻辑问题上的得分为58.8%,在伦理问题上的得分为60%。这意味着ChatGPT在逻辑问题上接近通过范围,并已跨越伦理问题的阈值。本文认为,ChatGPT和其他语言学习模型可以成为电子学习者的宝贵工具;然而,研究表明它们的准确性仍有提升空间。为了提高ChatGPT未来的表现,需要进一步研究以更好地理解它如何回答不同类型的问题。