Prosthetic Joint Infection (PJI) is a prevalent and severe complication characterized by high diagnostic challenges. Currently, a unified diagnostic standard incorporating both computed tomography (CT) images and numerical text data for PJI remains unestablished, owing to the substantial noise in CT images and the disparity in data volume between CT images and text data. This study introduces a diagnostic method, HGT, based on deep learning and multimodal techniques. It effectively merges features from CT scan images and patients' numerical text data via a Unidirectional Selective Attention (USA) mechanism and a graph convolutional network (GCN)-based feature fusion network. We evaluated the proposed method on a custom-built multimodal PJI dataset, assessing its performance through ablation experiments and interpretability evaluations. Our method achieved an accuracy (ACC) of 91.4\% and an area under the curve (AUC) of 95.9\%, outperforming recent multimodal approaches by 2.9\% in ACC and 2.2\% in AUC, with a parameter count of only 68M. Notably, the interpretability results highlighted our model's strong focus and localization capabilities at lesion sites. This proposed method could provide clinicians with additional diagnostic tools to enhance accuracy and efficiency in clinical practice.

翻译：假体周围关节感染（PJI）是一种常见且严重的并发症，其诊断具有高度挑战性。目前，由于CT图像存在显著噪声以及CT图像与文本数据之间的数据量差异，尚未建立结合计算机断层扫描（CT）图像与数值文本数据的统一PJI诊断标准。本研究提出一种基于深度学习和多模态技术的诊断方法HGT。该方法通过单向选择性注意力（USA）机制和基于图卷积网络（GCN）的特征融合网络，有效融合CT扫描图像与患者数值文本数据的特征。我们在自建的多模态PJI数据集上评估了该方法，并通过消融实验和可解释性评估对其性能进行了分析。该方法实现了91.4%的准确率（ACC）和95.9%的曲线下面积（AUC），在ACC和AUC上分别比近期多模态方法高出2.9%和2.2%，且参数量仅为68M。值得注意的是，可解释性结果凸显了模型在病灶部位强大的关注与定位能力。该方法可为临床医生提供额外的诊断工具，以提高临床实践中的准确性和效率。