This study aims to simulate real-world clinical scenarios to systematically evaluate the ability of Large Language Models (LLMs) to extract core medical information from patient chief complaints laden with noise and redundancy, and to verify whether they exhibit a functional decline analogous to Metabolic Dysfunction-Associated Steatotic Liver Disease (MASLD). We employed a cross-sectional analysis design based on standardized medical probes, selecting four mainstream LLMs as research subjects: GPT-4o, Gemini 2.5, DeepSeek 3.1, and Qwen3-Max. An evaluation system comprising twenty medical probes across five core dimensions was used to simulate a genuine clinical communication environment. All probes had gold-standard answers defined by clinical experts and were assessed via a double-blind, inverse rating scale by two independent clinicians. The results show that all tested models exhibited functional defects to varying degrees, with Qwen3-Max demonstrating the best overall performance and Gemini 2.5 the worst. Under conditions of extreme noise, most models experienced a functional collapse. Notably, GPT-4o made a severe misjudgment in the risk assessment for pulmonary embolism (PE) secondary to deep vein thrombosis (DVT). This research is the first to empirically confirm that LLMs exhibit features resembling metabolic dysfunction when processing clinical information, proposing the innovative concept of "AI-Metabolic Dysfunction-Associated Steatotic Liver Disease (AI-MASLD)". These findings offer a crucial safety warning for the application of Artificial Intelligence (AI) in healthcare, emphasizing that current LLMs must be used as auxiliary tools under human expert supervision, as there remains a significant gap between their theoretical knowledge and practical clinical application.


翻译:本研究旨在模拟真实临床场景,系统评估大型语言模型从充满噪声和冗余的患者主诉中提取核心医学信息的能力,并验证其是否表现出类似代谢功能障碍相关脂肪性肝病的功能衰退。我们采用基于标准化医学探针的横断面分析设计,选取了四种主流大型语言模型作为研究对象:GPT-4o、Gemini 2.5、DeepSeek 3.1 和 Qwen3-Max。构建了一个包含五个核心维度、共计二十个医学探针的评估体系,以模拟真实的临床沟通环境。所有探针均设有临床专家定义的金标准答案,并由两位独立临床医生采用双盲反向评分量表进行评估。结果显示,所有测试模型均表现出不同程度的功能缺陷,其中 Qwen3-Max 整体表现最佳,Gemini 2.5 表现最差。在极端噪声条件下,大多数模型出现功能崩溃。值得注意的是,GPT-4o 在深静脉血栓继发肺栓塞的风险评估中出现了严重误判。本研究首次通过实证证实,大型语言模型在处理临床信息时表现出类似代谢功能障碍的特征,并提出了“人工智能代谢功能障碍相关脂肪性肝病”的创新概念。这些发现为人工智能在医疗健康领域的应用提供了重要的安全警示,强调当前的大型语言模型必须在人类专家监督下作为辅助工具使用,其理论知识与实际临床应用之间仍存在显著差距。

0
下载
关闭预览

相关内容

LLM4SR:关于大规模语言模型在科学研究中的应用综述
专知会员服务
42+阅读 · 2025年1月9日
Single-Shot Object Detection with Enriched Semantics
统计学习与视觉计算组
14+阅读 · 2018年8月29日
国家自然科学基金
0+阅读 · 2015年12月31日
国家自然科学基金
46+阅读 · 2015年12月31日
VIP会员
相关基金
国家自然科学基金
0+阅读 · 2015年12月31日
国家自然科学基金
46+阅读 · 2015年12月31日
Top
微信扫码咨询专知VIP会员