Shapley values are model-agnostic methods for explaining model predictions. Many commonly used methods of computing Shapley values, known as off-manifold methods, rely on model evaluations on out-of-distribution input samples. Consequently, explanations obtained are sensitive to model behaviour outside the data distribution, which may be irrelevant for all practical purposes. While on-manifold methods have been proposed which do not suffer from this problem, we show that such methods are overly dependent on the input data distribution, and therefore result in unintuitive and misleading explanations. To circumvent these problems, we propose ManifoldShap, which respects the model's domain of validity by restricting model evaluations to the data manifold. We show, theoretically and empirically, that ManifoldShap is robust to off-manifold perturbations of the model and leads to more accurate and intuitive explanations than existing state-of-the-art Shapley methods.

翻译：夏普利值是用于解释模型预测的模型无关方法。许多常用的夏普利值计算方法（称为流形外方法）依赖于对分布外输入样本进行模型评估。因此，得到的解释对模型在数据分布之外的行为敏感，而这在实际应用中可能毫无意义。虽然已有不受此问题影响的流形上方法被提出，但我们证明这些方法过度依赖输入数据分布，从而导致反直觉且具有误导性的解释。为解决这些问题，我们提出ManifoldShap方法，该方法通过将模型评估限制在数据流形上来尊重模型的有效域。我们从理论和实验两方面证明，ManifoldShap对模型的流形外扰动具有鲁棒性，并且比现有最先进的夏普利方法能提供更准确且更直观的解释。