Influence estimation analyzes how changes to the training data can lead to different model predictions; this analysis can help us better understand these predictions, the models making those predictions, and the data sets they're trained on. However, most influence-estimation techniques are designed for deep learning models with continuous parameters. Gradient-boosted decision trees (GBDTs) are a powerful and widely-used class of models; however, these models are black boxes with opaque decision-making processes. In the pursuit of better understanding GBDT predictions and generally improving these models, we adapt recent and popular influence-estimation methods designed for deep learning models to GBDTs. Specifically, we adapt representer-point methods and TracIn, denoting our new methods TREX and BoostIn, respectively; source code is available at https://github.com/jjbrophy47/tree_influence. We compare these methods to LeafInfluence and other baselines using 5 different evaluation measures on 22 real-world data sets with 4 popular GBDT implementations. These experiments give us a comprehensive overview of how different approaches to influence estimation work in GBDT models. We find BoostIn is an efficient influence-estimation method for GBDTs that performs equally well or better than existing work while being four orders of magnitude faster. Our evaluation also suggests the gold-standard approach of leave-one-out~(LOO) retraining consistently identifies the single-most influential training example but performs poorly at finding the most influential set of training examples for a given target prediction.

翻译：影响估计分析训练数据的改变如何导致模型预测的不同；这种分析有助于我们更好地理解这些预测、做出这些预测的模型及其训练所用的数据集。然而，大多数影响估计技术是为具有连续参数的深度学习模型设计的。梯度提升决策树（GBDT）是一类功能强大且广泛使用的模型；然而，这些模型是决策过程不透明的黑箱模型。为更好地理解GBDT预测并普遍改进这些模型，我们将近期流行的、为深度学习模型设计的影响估计方法适配到GBDT上。具体而言，我们适配了表征点方法和TracIn，分别将其新方法命名为TREX和BoostIn；源代码见https://github.com/jjbrophy47/tree_influence。我们使用5种不同的评估指标，在22个真实世界数据集上，结合4种流行的GBDT实现，将这些方法与LeafInfluence及其他基线方法进行比较。这些实验全面展示了不同影响估计方法在GBDT模型中的表现。我们发现BoostIn是一种高效的GBDT影响估计方法，其性能与现有工作相当或更优，同时速度快四个数量级。我们的评估还表明，金标准方法——留一法（LOO）重训练——能一致地识别出最具单一样本影响力的训练样例，但在为给定目标预测寻找最具影响力的训练样例集合时表现较差。