Chest X-rays (CXRs) are the most frequently performed imaging test in clinical practice. Recent advances in the development of vision-language foundation models (FMs) give rise to the possibility of performing automated CXR interpretation, which can assist physicians with clinical decision-making and improve patient outcomes. However, developing FMs that can accurately interpret CXRs is challenging due to the (1) limited availability of large-scale vision-language datasets in the medical image domain, (2) lack of vision and language encoders that can capture the complexities of medical data, and (3) absence of evaluation frameworks for benchmarking the abilities of FMs on CXR interpretation. In this work, we address these challenges by first introducing \emph{CheXinstruct} - a large-scale instruction-tuning dataset curated from 28 publicly-available datasets. We then present \emph{CheXagent} - an instruction-tuned FM capable of analyzing and summarizing CXRs. To build CheXagent, we design a clinical large language model (LLM) for parsing radiology reports, a vision encoder for representing CXR images, and a network to bridge the vision and language modalities. Finally, we introduce \emph{CheXbench} - a novel benchmark designed to systematically evaluate FMs across 8 clinically-relevant CXR interpretation tasks. Extensive quantitative evaluations and qualitative reviews with five expert radiologists demonstrate that CheXagent outperforms previously-developed general- and medical-domain FMs on CheXbench tasks. Furthermore, in an effort to improve model transparency, we perform a fairness evaluation across factors of sex, race and age to highlight potential performance disparities. Our project is at \url{https://stanford-aimi.github.io/chexagent.html}.

翻译：胸部X光检查（CXR）是临床实践中最常用的影像学检查。近期，视觉语言基础模型的发展为自动化CXR解读提供了可能，这有助于辅助医生进行临床决策并改善患者预后。然而，开发能准确解读CXR的基础模型面临三大挑战：（1）医学影像领域缺乏大规模视觉语言数据集；（2）缺乏能捕捉医学数据复杂性的视觉和语言编码器；（3）缺乏用于评估基础模型CXR解读能力的基准框架。本研究通过以下方式解决这些挑战：首先提出CheXinstruct——一个从28个公开数据集中整理的大规模指令调优数据集；随后提出CheXagent——一个经指令调优、能分析并总结CXR结果的基础模型。为构建CheXagent，我们设计了用于解析放射报告的临床大语言模型、用于表征CXR图像的视觉编码器，以及连接视觉与语言模态的网络。最后，我们引入CheXbench——一个专门设计用于系统评估基础模型在8项临床相关CXR解读任务中表现的新基准。大量定量评估及五位放射科专家进行的定性审查表明，在CheXbench任务中，CheXagent优于此前开发的通用及医学领域基础模型。此外，为提升模型透明度，我们按性别、种族和年龄进行了公平性评估，以揭示潜在的性能差异。项目主页：https://stanford-aimi.github.io/chexagent.html。