Gradient-boosted decision trees (GBDT) are widely used and highly effective machine learning approach for tabular data modeling. However, their complex structure may lead to low robustness against small covariate perturbation in unseen data. In this study, we apply one-hot encoding to convert a GBDT model into a linear framework, through encoding of each tree leaf to one dummy variable. This allows for the use of linear regression techniques, plus a novel risk decomposition for assessing the robustness of a GBDT model against covariate perturbations. We propose to enhance the robustness of GBDT models by refitting their linear regression forms with $L_1$ or $L_2$ regularization. Theoretical results are obtained about the effect of regularization on the model performance and robustness. It is demonstrated through numerical experiments that the proposed regularization approach can enhance the robustness of the one-hot-encoded GBDT models.

翻译：梯度提升决策树（GBDT）是一种广泛应用于表格数据建模且高效性突出的机器学习方法。然而，其复杂结构可能导致对未见数据中微小协变量扰动的鲁棒性较低。本研究通过将每棵树的叶节点编码为一个哑变量，对GBDT模型施加独热编码以将其转化为线性框架。这使我们能够利用线性回归技术，并结合一种新的风险分解方法，来评估GBDT模型对协变量扰动的鲁棒性。我们提出通过使用$L_1$或$L_2$正则化对GBDT模型的线性回归形式进行重拟合，以增强其鲁棒性。理论结果揭示了正则化对模型性能与鲁棒性的影响。数值实验证明，所提出的正则化方法能够有效提升经独热编码的GBDT模型的鲁棒性。