Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of annotated data, which often is scarce in the medical domain. Zero-shot methods address this challenge by allowing a flexible adaption to new settings with different clinical findings without relying on labeled data. Further, to integrate automated diagnosis in the clinical workflow, methods should be transparent and explainable, increasing medical professionals' trust and facilitating correctness verification. In this work, we introduce Xplainer, a novel framework for explainable zero-shot diagnosis in the clinical setting. Xplainer adapts the classification-by-description approach of contrastive vision-language models to the multi-label medical diagnosis task. Specifically, instead of directly predicting a diagnosis, we prompt the model to classify the existence of descriptive observations, which a radiologist would look for on an X-Ray scan, and use the descriptor probabilities to estimate the likelihood of a diagnosis. Our model is explainable by design, as the final diagnosis prediction is directly based on the prediction of the underlying descriptors. We evaluate Xplainer on two chest X-ray datasets, CheXpert and ChestX-ray14, and demonstrate its effectiveness in improving the performance and explainability of zero-shot diagnosis. Our results suggest that Xplainer provides a more detailed understanding of the decision-making process and can be a valuable tool for clinical diagnosis.

翻译：从医学图像中自动诊断预测是支持临床决策的重要工具。然而，这类系统通常需要依赖大量标注数据进行训练，在医学领域这种数据往往稀缺。零样本方法通过允许灵活适应不同临床发现的新场景，无需依赖标注数据，从而解决了这一挑战。此外，为了将自动诊断整合到临床工作流中，方法应具备透明性和可解释性，以增强医疗专业人士的信任并促进正确性验证。在本工作中，我们提出了Xplainer，一种用于临床场景中可解释零样本诊断的新型框架。Xplainer将对比视觉语言模型的“分类-通过-描述”方法适配到多标签医学诊断任务中。具体来说，我们不是直接预测诊断结果，而是促使模型对放射科医生在X射线扫描中会寻找的描述性观测的存在性进行分类，并利用描述符概率来估计诊断的可能性。我们的模型在设计上具有可解释性，因为最终诊断预测直接基于底层描述符的预测。我们在两个胸部X射线数据集CheXpert和ChestX-ray14上评估了Xplainer，并证明了其在提升零样本诊断性能和可解释性方面的有效性。我们的结果表明，Xplainer提供了对决策过程更详细的理解，并可能成为临床诊断的宝贵工具。