The extraction of structured clinical information from free-text radiology reports in the form of radiology graphs has been demonstrated to be a valuable approach for evaluating the clinical correctness of report-generation methods. However, the direct generation of radiology graphs from chest X-ray (CXR) images has not been attempted. To address this gap, we propose a novel approach called Prior-RadGraphFormer that utilizes a transformer model with prior knowledge in the form of a probabilistic knowledge graph (PKG) to generate radiology graphs directly from CXR images. The PKG models the statistical relationship between radiology entities, including anatomical structures and medical observations. This additional contextual information enhances the accuracy of entity and relation extraction. The generated radiology graphs can be applied to various downstream tasks, such as free-text or structured reports generation and multi-label classification of pathologies. Our approach represents a promising method for generating radiology graphs directly from CXR images, and has significant potential for improving medical image analysis and clinical decision-making.

翻译：从自由文本放射学报告中提取结构化临床信息并构建放射学图谱，已被证明是评估报告生成方法临床准确性的有效途径。然而，直接从胸部X光片（CXR）图像生成放射学图谱的研究尚属空白。为弥补这一不足，我们提出了一种名为Prior-RadGraphFormer的新方法，该方法采用带有先验知识的Transformer模型——以概率知识图谱（PKG）的形式——直接从CXR图像生成放射学图谱。PKG建模了放射学实体（包括解剖结构和医学观察）之间的统计关系，这种额外的上下文信息增强了实体与关系抽取的准确性。生成的放射学图谱可应用于多种下游任务，例如自由文本或结构化报告生成以及病理多标签分类。我们的方法为直接从CXR图像生成放射学图谱提供了一种有前景的方案，在改善医学图像分析和临床决策方面具有巨大潜力。