We focus on modeling the relationship between an input feature vector and the predicted outcome of a trained decision tree using mixed-integer optimization. This can be used in many practical applications where a decision tree or tree ensemble is incorporated into an optimization problem to model the predicted outcomes of a decision. We propose tighter mixed-integer optimization formulations than those previously introduced. Existing formulations can be shown to have linear relaxations that have fractional extreme points, even for the simple case of modeling a single decision tree. A formulation we propose, based on a projected union of polyhedra approach, is ideal for a single decision tree. While the formulation is generally not ideal for tree ensembles or if additional constraints are added, it generally has fewer extreme points, leading to a faster time to solve, particularly if the formulation has relatively few trees. However, previous work has shown that formulations based on a binary representation of the feature vector perform well computationally and hence are attractive for use in practical applications. We present multiple approaches to tighten existing formulations with binary vectors, and show that fractional extreme points are removed when there are multiple splits on the same feature. At an extreme, we prove that this results in ideal formulations for tree ensembles modeling a one-dimensional feature vector. Building on this result, we also show via numerical simulations that these additional constraints result in significantly tighter linear relaxations when the feature vector is low dimensional. We also present instances where the time to solve to optimality is significantly improved using these formulations.

翻译：我们聚焦于使用混合整数优化来建模输入特征向量与经过训练的决策树预测输出之间的关系。这一方法可应用于诸多实际场景，其中决策树或树集成被纳入优化问题，以建模决策的预测结果。我们提出了比已有表述更紧致的混合整数优化形式。已有表述的线性松弛存在分数极值点，即使是建模单棵决策树的简单情况也不例外。我们提出的一种基于多面体投影并集的方法，对于单棵决策树是理想的。尽管该表述在树集成或添加额外约束时通常不具理想性，但其极值点数量较少，从而缩短求解时间，尤其在树数量较少的情况下表现突出。然而，先前工作表明，基于特征向量二进制表示的表述在计算上表现优异，因此在实践中具有吸引力。我们提出了多种方法来收紧现有的二进制向量表述，并证明当同一特征存在多个分裂时，分数极值点得以消除。在极端情况下，我们证明这能为一维特征向量的树集成模型生成理想表述。基于此结果，我们还通过数值模拟表明，当特征向量维度较低时，这些额外约束能显著收紧线性松弛。此外，我们展示了使用这些表述可将最优解求解时间显著缩短的实例。