Purpose: To investigate chest radiograph (CXR) classification performance of vision transformers (ViT) and interpretability of attention-based saliency using the example of pneumothorax classification. Materials and Methods: In this retrospective study, ViTs were fine-tuned for lung disease classification using four public data sets: CheXpert, Chest X-Ray 14, MIMIC CXR, and VinBigData. Saliency maps were generated using transformer multimodal explainability and gradient-weighted class activation mapping (GradCAM). Classification performance was evaluated on the Chest X-Ray 14, VinBigData, and SIIM-ACR data sets using the area under the receiver operating characteristic curve analysis (AUC) and compared with convolutional neural networks (CNNs). The explainability methods were evaluated with positive/negative perturbation, sensitivity-n, effective heat ratio, intra-architecture repeatability and interarchitecture reproducibility. In the user study, three radiologists classified 160 CXRs with/without saliency maps for pneumothorax and rated their usefulness. Results: ViTs had comparable CXR classification AUCs compared with state-of-the-art CNNs 0.95 (95% CI: 0.943, 0.950) versus 0.83 (95%, CI 0.826, 0.842) on Chest X-Ray 14, 0.84 (95% CI: 0.769, 0.912) versus 0.83 (95% CI: 0.760, 0.895) on VinBigData, and 0.85 (95% CI: 0.847, 0.861) versus 0.87 (95% CI: 0.868, 0.882) on SIIM ACR. Both saliency map methods unveiled a strong bias toward pneumothorax tubes in the models. Radiologists found 47% of the attention-based saliency maps useful and 39% of GradCAM. The attention-based methods outperformed GradCAM on all metrics. Conclusion: ViTs performed similarly to CNNs in CXR classification, and their attention-based saliency maps were more useful to radiologists and outperformed GradCAM.

翻译：目的：以气胸分类为例，探究视觉Transformer在胸部X光片分类中的性能及基于注意力的显著性图的可解释性。材料与方法：在本回顾性研究中，使用四个公开数据集（CheXpert、Chest X-Ray 14、MIMIC CXR和VinBigData）对视觉Transformer进行肺部疾病分类的微调。通过Transformer多模态可解释性方法和梯度加权类激活映射生成显著性图。分类性能在Chest X-Ray 14、VinBigData和SIIM-ACR数据集上使用受试者工作特征曲线下面积（AUC）进行评估，并与卷积神经网络进行比较。通过正/负扰动、敏感性-n、有效热图比率、架构内重复性和架构间可重复性对可解释性方法进行评估。在用户研究中，三位放射科医师对160张胸部X光片（使用/不使用针对气胸的显著性图）进行分类，并评估其有用性。结果：与最先进的CNN相比，视觉Transformer在胸部X光片分类上具有相当的AUC值：在Chest X-Ray 14上为0.95（95%置信区间：0.943，0.950）对比0.83（95%置信区间：0.826，0.842），在VinBigData上为0.84（95%置信区间：0.769，0.912）对比0.83（95%置信区间：0.760，0.895），在SIIM ACR上为0.85（95%置信区间：0.847，0.861）对比0.87（95%置信区间：0.868，0.882）。两种显著性图方法均揭示模型对气胸管存在显著偏向。放射科医师认为47%的基于注意力的显著性图有用，而GradCAM为39%。基于注意力的方法在所有指标上均优于GradCAM。结论：视觉Transformer在胸部X光片分类中性能与CNN相当，且其基于注意力的显著性图对放射科医师更具实用性且优于GradCAM。