Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of annotated data, which often is scarce in the medical domain. Zero-shot methods address this challenge by allowing a flexible adaption to new settings with different clinical findings without relying on labeled data. Further, to integrate automated diagnosis in the clinical workflow, methods should be transparent and explainable, increasing medical professionals' trust and facilitating correctness verification. In this work, we introduce Xplainer, a novel framework for explainable zero-shot diagnosis in the clinical setting. Xplainer adapts the classification-by-description approach of contrastive vision-language models to the multi-label medical diagnosis task. Specifically, instead of directly predicting a diagnosis, we prompt the model to classify the existence of descriptive observations, which a radiologist would look for on an X-Ray scan, and use the descriptor probabilities to estimate the likelihood of a diagnosis. Our model is explainable by design, as the final diagnosis prediction is directly based on the prediction of the underlying descriptors. We evaluate Xplainer on two chest X-ray datasets, CheXpert and ChestX-ray14, and demonstrate its effectiveness in improving the performance and explainability of zero-shot diagnosis. Our results suggest that Xplainer provides a more detailed understanding of the decision-making process and can be a valuable tool for clinical diagnosis.

翻译：从医学图像中进行自动诊断预测是支持临床决策的重要工具。然而，此类系统通常需要大量标注数据进行训练，而医学领域的数据往往稀缺。零样本方法通过无需依赖标注数据即可灵活适应具有不同临床发现的新场景，从而解决了这一挑战。此外，为将自动诊断整合到临床工作流程中，方法需具备透明性和可解释性，这能增强医疗专业人员的信任并促进正确性验证。本文提出了Xplainer——一种用于临床环境中可解释零样本诊断的新型框架。Xplainer将对比视觉-语言模型的“基于描述分类”方法适配至多标签医学诊断任务。具体而言，我们不直接预测诊断结果，而是提示模型对放射科医生在X光扫描中应观察到的描述性征象进行分类，并利用描述符概率来估计诊断的可能性。我们的模型在设计上具有可解释性，因为最终诊断预测直接基于底层描述符的预测。我们在两个胸部X光数据集CheXpert和ChestX-ray14上评估了Xplainer，证明了其在提升零样本诊断性能与可解释性方面的有效性。结果表明，Xplainer能更细致地理解决策过程，并可成为临床诊断中的有价值工具。