The ability to remove features from the input of machine learning models is very important to understand and interpret model predictions. However, this is non-trivial for vision models since masking out parts of the input image typically causes large distribution shifts. This is because the baseline color used for masking (typically grey or black) is out of distribution. Furthermore, the shape of the mask itself can contain unwanted signals which can be used by the model for its predictions. Recently, there has been some progress in mitigating this issue (called missingness bias) in image masking for vision transformers. In this work, we propose a new masking method for CNNs we call layer masking in which the missingness bias caused by masking is reduced to a large extent. Intuitively, layer masking applies a mask to intermediate activation maps so that the model only processes the unmasked input. We show that our method (i) is able to eliminate or minimize the influence of the mask shape or color on the output of the model, and (ii) is much better than replacing the masked region by black or grey for input perturbation based interpretability techniques like LIME. Thus, layer masking is much less affected by missingness bias than other masking strategies. We also demonstrate how the shape of the mask may leak information about the class, thus affecting estimates of model reliance on class-relevant features derived from input masking. Furthermore, we discuss the role of data augmentation techniques for tackling this problem, and argue that they are not sufficient for preventing model reliance on mask shape. The code for this project is publicly available at https://github.com/SriramB-98/layer_masking

翻译：从机器学习模型输入中移除特征的能力对于理解和解释模型预测至关重要。然而，对于视觉模型而言，这一任务颇具挑战性，因为遮蔽输入图像的局部区域通常会导致严重的分布偏移。这是由于掩码所使用的基准颜色（通常为灰色或黑色）偏离了数据分布。此外，掩码本身的形状可能包含模型可用于预测的无关信号。近期，在缓解视觉Transformer图像掩码中的这一问题（称为缺失偏差）方面取得了一定进展。本文针对卷积神经网络提出了一种新的掩码方法——层级掩码，该方法能大幅降低由掩码引起的缺失偏差。直观而言，层级掩码对中间激活图施加掩码，使模型仅处理未被遮蔽的输入。我们证明该方法能够：（i）消除或最小化掩码形状或颜色对模型输出的影响；（ii）在基于输入扰动的可解释性技术（如LIME）中，其效果远优于使用黑色或灰色替换被遮蔽区域。因此，层级掩码受缺失偏差的影响远小于其他掩码策略。我们还展示了掩码形状可能泄露类别信息，从而影响基于输入掩码评估模型对类别相关特征依赖度的结果。此外，我们讨论了数据增强技术在解决该问题中的作用，并论证其不足以防止模型依赖掩码形状。该项目代码已公开于https://github.com/SriramB-98/layer_masking。