The ability to remove features from the input of machine learning models is very important to understand and interpret model predictions. However, this is non-trivial for vision models since masking out parts of the input image typically causes large distribution shifts. This is because the baseline color used for masking (typically grey or black) is out of distribution. Furthermore, the shape of the mask itself can contain unwanted signals which can be used by the model for its predictions. Recently, there has been some progress in mitigating this issue (called missingness bias) in image masking for vision transformers. In this work, we propose a new masking method for CNNs we call layer masking in which the missingness bias caused by masking is reduced to a large extent. Intuitively, layer masking applies a mask to intermediate activation maps so that the model only processes the unmasked input. We show that our method (i) is able to eliminate or minimize the influence of the mask shape or color on the output of the model, and (ii) is much better than replacing the masked region by black or grey for input perturbation based interpretability techniques like LIME. Thus, layer masking is much less affected by missingness bias than other masking strategies. We also demonstrate how the shape of the mask may leak information about the class, thus affecting estimates of model reliance on class-relevant features derived from input masking. Furthermore, we discuss the role of data augmentation techniques for tackling this problem, and argue that they are not sufficient for preventing model reliance on mask shape. The code for this project is publicly available at https://github.com/SriramB-98/layer_masking

翻译：从机器学习模型输入中移除特征的能力对于理解和解释模型预测至关重要。然而，对于视觉模型而言，这一任务具有非平凡性，因为遮蔽输入图像的部分区域通常会导致较大的分布偏移。这是由于用于掩码的基准颜色（通常为灰色或黑色）处于分布外。此外，掩码本身的形状可能包含模型可用于预测的非期望信号。近期，针对视觉Transformer的图像掩码中缓解此问题（称为缺失偏差）的研究已取得一定进展。在本工作中，我们提出一种针对卷积神经网络的新掩码方法——层掩码，该方法能够大幅降低由掩码引起的缺失偏差。直观而言，层掩码对中间激活图施加掩码，使模型仅处理未被遮蔽的输入。我们证明，该方法（i）能消除或最小化掩码形状或颜色对模型输出的影响，且（ii）在基于输入扰动的可解释性技术（如LIME）中，显著优于使用黑色或灰色替代掩码区域的方法。因此，层掩码受缺失偏差的影响远小于其他掩码策略。我们还展示了掩码形状可能泄露类别信息，从而影响通过输入掩码推导的模型对类别相关特征依赖度的估计。此外，我们讨论了数据增强技术在解决该问题中的作用，并论证其不足以阻止模型对掩码形状的依赖。本项目的代码已公开于https://github.com/SriramB-98/layer_masking。