Global feature effects such as partial dependence (PD) and accumulated local effects (ALE) plots are widely used to interpret black-box models. However, they are only estimates of true underlying effects, and their reliability depends on multiple sources of error. Despite the popularity of global feature effects, these error sources are largely unexplored. In particular, the practically relevant question of whether to use training or holdout data to estimate feature effects remains unanswered. We address this gap by providing a systematic, estimator-level analysis that disentangles sources of bias and variance for PD and ALE. To this end, we derive a mean-squared-error decomposition that separates model bias, estimation bias, model variance, and estimation variance, and analyze their dependence on model characteristics, data selection, and sample size. We validate our theoretical findings through an extensive simulation study across multiple data-generating processes, learners, estimation strategies (training data, validation data, and cross-validation), and sample sizes. Our results reveal that, while using holdout data is theoretically the cleanest, potential biases arising from the training data are empirically negligible and dominated by the impact of the usually higher sample size. The estimation variance depends on both the presence of interactions and the sample size, with ALE being particularly sensitive to the latter. Cross-validation-based estimation is a promising approach that reduces the model variance component, particularly for overfitting models. Our analysis provides a principled explanation of the sources of error in feature effect estimates and offers concrete guidance on choosing estimation strategies when interpreting machine learning models.

翻译：全局特征效应，如部分依赖（PD）图和累积局部效应（ALE）图，被广泛用于解释黑盒模型。然而，它们仅是真实潜在效应的估计值，其可靠性取决于多种误差源。尽管全局特征效应广受欢迎，但这些误差源在很大程度上尚未被充分探究。特别是，一个实践中重要的问题——应使用训练数据还是留出数据来估计特征效应——仍未得到解答。我们通过提供一个系统性的、估计器层面的分析来填补这一空白，该分析解构了PD和ALE的偏差与方差来源。为此，我们推导了一个均方误差分解，将模型偏差、估计偏差、模型方差和估计方差分离开来，并分析了它们对模型特性、数据选择和样本量的依赖性。我们通过一项广泛的模拟研究验证了我们的理论发现，该研究涵盖了多种数据生成过程、学习器、估计策略（训练数据、验证数据和交叉验证）以及样本量。我们的结果表明，虽然理论上使用留出数据最为清晰，但由训练数据引起的潜在偏差在经验上可忽略不计，且通常被更大的样本量所带来的影响所主导。估计方差既取决于交互作用的存在，也取决于样本量，其中ALE对后者尤为敏感。基于交叉验证的估计是一种有前景的方法，它能减少模型方差分量，特别是对于过拟合模型。我们的分析为特征效应估计中的误差来源提供了原理性解释，并为解释机器学习模型时选择估计策略提供了具体指导。