Tree ensembles achieve state-of-the-art performance despite being greedily optimized. Global refinement (GR) reduces greediness by jointly and globally optimizing all constant leaves. We propose Joint Optimization of Piecewise Linear ENsembles (JOPLEN), a piecewise-linear extension of GR. Compared to GR, JOPLEN improves model flexibility and can apply common penalties, including sparsity-promoting matrix norms and subspace-norms, to nonlinear prediction. We evaluate the Frobenius norm, $\ell_{2,1}$ norm, and Laplacian regularization for 146 regression and classification datasets; JOPLEN, combined with GB trees and RF, achieves superior performance in both settings. Additionally, JOPLEN with a nuclear norm penalty empirically learns smooth and subspace-aligned functions. Finally, we perform multitask feature selection by extending the Dirty LASSO. JOPLEN Dirty LASSO achieves a superior feature sparsity/performance tradeoff to linear and gradient boosted approaches. We anticipate that JOPLEN will improve regression, classification, and feature selection across many fields.

翻译：尽管树集成模型是通过贪心优化训练的，但其性能仍能达到最先进水平。全局优化（GR）通过对所有常数叶节点进行联合全局优化来减少贪心性。我们提出分段线性集成联合优化方法（JOPLEN），这是GR的分段线性扩展。与GR相比，JOPLEN提升了模型灵活性，并可对非线性预测施加包括稀疏性诱导矩阵范数和子空间范数在内的常见惩罚项。我们利用Frobenius范数、$\ell_{2,1}$范数和拉普拉斯正则化在146个回归与分类数据集上进行了评估；结合梯度提升树（GB）和随机森林（RF）的JOPLEN在两种任务中均展现出更优性能。此外，采用核范数惩罚的JOPLEN通过经验学习得到了平滑且对齐子空间的函数。最后，我们通过扩展Dirty LASSO实现多任务特征选择。与线性及梯度提升方法相比，JOPLEN Dirty LASSO在特征稀疏性与性能权衡方面表现更优。我们预期JOPLEN将推动多个领域的回归、分类及特征选择研究。