Despite the success of deep learning for text and image data, tree-based ensemble models are still state-of-the-art for machine learning with heterogeneous tabular data. However, there is a significant need for tabular-specific gradient-based methods due to their high flexibility. In this paper, we propose $\text{GRANDE}$, $\text{GRA}$die$\text{N}$t-Based $\text{D}$ecision Tree $\text{E}$nsembles, a novel approach for learning hard, axis-aligned decision tree ensembles using end-to-end gradient descent. GRANDE is based on a dense representation of tree ensembles, which affords to use backpropagation with a straight-through operator to jointly optimize all model parameters. Our method combines axis-aligned splits, which is a useful inductive bias for tabular data, with the flexibility of gradient-based optimization. Furthermore, we introduce an advanced instance-wise weighting that facilitates learning representations for both, simple and complex relations, within a single model. We conducted an extensive evaluation on a predefined benchmark with 19 classification datasets and demonstrate that our method outperforms existing gradient-boosting and deep learning frameworks on most datasets. The method is available under: https://github.com/s-marton/GRANDE

翻译：尽管深度学习在文本和图像数据领域取得了成功，基于树的集成模型在处理异构表格数据的机器学习任务中仍处于领先地位。然而，由于梯度方法具有高度灵活性，针对表格数据的专用梯度方法存在显著需求。本文提出 $\text{GRANDE}$（$\text{GRA}$die$\text{N}$t-Based $\text{D}$ecision Tree $\text{E}$nsembles），一种通过端到端梯度下降学习硬性、轴对齐决策树集成的新方法。GRANDE 基于树集成的稠密表示，可利用直通算子进行反向传播，实现所有模型参数的联合优化。该方法将轴对齐分割（表格数据的重要归纳偏置）与梯度优化的灵活性相结合。此外，我们引入先进的实例级加权机制，使单个模型能够同时学习简单与复杂关系的表征。在包含19个分类数据集的预定义基准测试中，我们进行了广泛评估，证明该方法在大多数数据集上优于现有梯度提升和深度学习框架。代码已开源：https://github.com/s-marton/GRANDE