Feature removal is a central building block for eXplainable AI (XAI), both for occlusion-based explanations (Shapley values) as well as their evaluation (pixel flipping, PF). However, occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. This ambiguity limits the usefulness of occlusion-based approaches. For example, PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF). This study proposes two complementary perspectives to resolve this disagreement problem. Firstly, we address the common criticism of occlusion-based XAI, that artificial samples lead to unreliable model evaluations. We propose to measure the reliability by the R(eference)-Out-of-Model-Scope (OMS) score. The R-OMS score enables a systematic comparison of occlusion strategies and resolves the disagreement problem by grouping consistent PF rankings. Secondly, we show that the insightfulness of MIF and LIF is conversely dependent on the R-OMS score. To leverage this, we combine the MIF and LIF measures into the symmetric relevance gain (SRG) measure. This breaks the inherent connection to the underlying occlusion strategy and leads to consistent rankings. This resolves the disagreement problem, which we verify for a set of 40 different occlusion strategies.

翻译：特征移除是可解释人工智能（XAI）的核心构建模块，既用于基于遮盖的解释（Shapley值），也用于其评估（像素翻转，PF）。然而，遮盖策略差异显著，从简单的均值替换到使用最先进扩散模型进行图像修复，不一而足。这种歧义限制了基于遮盖方法的有用性。例如，PF基准测试导致相互矛盾的排序。这种矛盾因竞争的PF指标而加剧：特征移除要么从最具影响力到最不具影响力（MIF），要么从最不具影响力到最具影响力（LIF）。本研究提出两种互补视角来解决这一分歧问题。首先，我们针对基于遮盖的XAI的常见批评——即人工样本导致不可靠的模型评估——提出通过参考超出模型范围（R-OMS）分数来衡量可靠性。R-OMS分数能够系统比较遮盖策略，并通过分组一致的PF排序来解决分歧问题。其次，我们证明MIF和LIF的洞察力与R-OMS分数呈反向关系。为利用这一点，我们将MIF和LIF指标结合成对称相关性增益（SRG）指标。这打破了与底层遮盖策略的内在联系，并产生一致的排序。这解决了分歧问题，我们通过40种不同遮盖策略进行了验证。