When applied to Image-to-text models, interpretability methods often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence. Those explanations are expensive to compute and unable to comprehensively explain the model's output. Therefore, these models often require some sort of approximation that eventually leads to misleading explanations. We develop a framework based on SHAP, that allows for generating comprehensive, meaningful explanations leveraging the meaning representation of the output sequence as a whole. Moreover, by exploiting semantic priors in the visual backbone, we extract an arbitrary number of features that allows the efficient computation of Shapley values on large-scale models, generating at the same time highly meaningful visual explanations. We demonstrate that our method generates semantically more expressive explanations than traditional methods at a lower compute cost and that it can be generalized over other explainability methods.

翻译：当可解释性方法应用于图像到文本模型时，通常提供逐token的解释——即为生成序列中的每个token计算一个视觉解释。这些解释计算成本高昂，且无法全面解释模型的输出。因此，这类模型往往需要某种形式的近似，最终导致产生具有误导性的解释。我们开发了一个基于SHAP的框架，能够利用输出序列整体的含义表示，生成全面且有意义的解释。此外，通过利用视觉骨干网络中的语义先验，我们提取任意数量的特征，从而能够高效计算大规模模型上的Shapley值，同时生成高度有意义的视觉解释。我们证明，与传统方法相比，我们的方法在更低计算成本下生成了语义上更具表现力的解释，并且可以泛化到其他可解释性方法上。