Gradient boosted decision trees require a stopping rule to avoid overfitting. The standard rule monitors a validation loss and stops if the loss fails to improve for a fixed patience period. However, the patience parameter has no interpretable scale and validation losses can be noisy or implicitly defined by a user-specified gradient. We propose ScoreStop, a gradient-based early-stopping rule that casts the stopping decision at each iteration as a test of the null hypothesis that the current predictor is the population risk minimizer. We use a functional score test, computed on validation data, with a statistic that is scale-invariant in the update direction, with a known asymptotic distribution under the null. Because our test uses gradients rather than loss values, the same construction applies to implicit losses such as LambdaRank, and data-dependent losses such as Cox regression via influence functions. In synthetic experiments and real-data benchmarks, we show that ScoreStop is competitive with loss-based methods.

翻译：梯度提升决策树需要一种停止规则以避免过拟合。标准规则通过监控验证损失并在损失持续未改善时达到固定耐心期后停止训练。然而，耐心参数缺乏可解释的尺度，且验证损失可能存在噪声，或由用户指定的梯度隐式定义。我们提出ScoreStop，一种基于梯度的早期停止规则，它将每次迭代的停止决策转化为对当前预测器是否为总体风险最小化者的原假设检验。我们利用验证数据计算函数得分检验，其统计量在更新方向上具有尺度不变性，并在原假设下具有已知的渐近分布。由于我们的检验使用梯度而非损失值，相同的构造同样适用于隐式损失（如LambdaRank）以及通过影响函数定义的数据依赖损失（如Cox回归）。在合成实验和真实数据基准测试中，我们证明ScoreStop与基于损失的方法具有竞争力。