SHAP is a popular method for measuring variable importance in machine learning models. In this paper, we study the algorithm used to estimate SHAP scores and show that it is a transformation of the functional ANOVA decomposition. We use this connection to show that challenges in SHAP approximations largely relate to the choice of a feature distribution and the number of $2^p$ ANOVA terms estimated. We argue that the connection between machine learning explainability and sensitivity analysis is illuminating in this case, but the immediate practical consequences are not obvious since the two fields face a different set of constraints. Machine learning explainability concerns models which are inexpensive to evaluate but often have hundreds, if not thousands, of features. Sensitivity analysis typically deals with models from physics or engineering which may be very time consuming to run, but operate on a comparatively small space of inputs.

翻译：SHAP是衡量机器学习模型中不同重要性的流行方法。 在本文中,我们研究了用于估算 SHAP 分数的算法,并表明这是功能性 ANOVA分解的转变。我们利用这一连接来表明, SHAP 近似中的挑战主要与地貌分布的选择和估计的2 ⁇ p$ ANOVA 术语数有关。我们争辩说,机器学习解释和敏感性分析之间的联系在本案中是很有启发性的,但直接的实际后果并不明显,因为这两个领域面临一系列不同的限制。 机器学习解释涉及一些价格低廉、但往往有数百(如果不是数千)个特征的模型。 敏感性分析通常涉及物理或工程模型,这些模型运行的时间可能非常耗时,但在相对较小的投入空间上运行。