Existing tools for explaining the output of image classifiers can be divided into white-box, which rely on access to the model internals, and black-box, agnostic to the model. As the usage of AI in the medical domain grows, so too does the usage of explainability tools. Existing work on medical image explanations focuses on white-box tools, such as gradcam. However, there are clear advantages to switching to a black-box tool, including the ability to use it with any classifier and the wide selection of black-box tools available. On standard images, black-box tools are as precise as white-box. In this paper we compare the performance of several black-box methods against gradcam on a brain cancer MRI dataset. We demonstrate that most black-box tools are not suitable for explaining medical image classifications and present a detailed analysis of the reasons for their shortcomings. We also show that one black-box tool, a causal explainability-based rex, performs as well as \gradcam.

翻译：现有图像分类器输出解释工具可分为白盒工具（依赖模型内部访问权限）与黑盒工具（与模型无关）。随着人工智能在医学领域的应用日益广泛，可解释性工具的使用也随之增加。现有医学图像解释研究主要聚焦于白盒工具（如GradCAM）。然而，转向黑盒工具具有明显优势，包括可适配任意分类器以及丰富的黑盒工具选择。在标准图像上，黑盒工具与白盒工具具有同等精度。本文在脑癌MRI数据集上比较了多种黑盒方法与GradCAM的性能表现。研究表明大多数黑盒工具不适用于医学图像分类解释，并详细分析了其缺陷成因。同时证明基于因果可解释性的Rex工具与GradCAM性能相当。