Random Forests (RF) and Extreme Gradient Boosting (XGBoost) are two of the most widely used and highly performing classification and regression models. They aggregate equally weighted CART trees, generated randomly in RF or sequentially in XGBoost. In this paper, we propose Adaptive Forests (AF), a novel approach that adaptively selects the weights of the underlying CART models. AF combines (a) the Optimal Predictive-Policy Trees (OP2T) framework to prescribe tailored, input-dependent unequal weights to trees and (b) Mixed Integer Optimization (MIO) to refine weight candidates dynamically, enhancing overall performance. We demonstrate that AF consistently outperforms RF, XGBoost, and other weighted RF in binary and multi-class classification problems over 20+ real-world datasets.

翻译：随机森林（RF）与极端梯度提升（XGBoost）是当前应用最广泛且性能最优异的分类与回归模型之一。它们通过聚合等权重的分类回归树（CART）实现预测，其中RF以随机方式生成树，而XGBoost以序列化方式生成树。本文提出自适应森林（AF）这一创新方法，能够自适应地选择底层CART模型的权重。AF融合了（a）最优预测策略树（OP2T）框架——该框架可为各树分配定制化的、输入依赖的非等权重，以及（b）混合整数优化（MIO）方法——通过动态优化权重候选值以提升整体性能。我们在超过20个真实数据集上的二分类与多分类问题中验证了AF方法，结果表明其性能持续优于RF、XGBoost及其他加权随机森林模型。