The interpretability of models has become a crucial issue in Machine Learning because of algorithmic decisions' growing impact on real-world applications. Tree ensemble methods, such as Random Forests or XgBoost, are powerful learning tools for classification tasks. However, while combining multiple trees may provide higher prediction quality than a single one, it sacrifices the interpretability property resulting in "black-box" models. In light of this, we aim to develop an interpretable representation of a tree-ensemble model that can provide valuable insights into its behavior. First, given a target tree-ensemble model, we develop a hierarchical visualization tool based on a heatmap representation of the forest's feature use, considering the frequency of a feature and the level at which it is selected as an indicator of importance. Next, we propose a mixed-integer linear programming (MILP) formulation for constructing a single optimal multivariate tree that accurately mimics the target model predictions. The goal is to provide an interpretable surrogate model based on oblique hyperplane splits, which uses only the most relevant features according to the defined forest's importance indicators. The MILP model includes a penalty on feature selection based on their frequency in the forest to further induce sparsity of the splits. The natural formulation has been strengthened to improve the computational performance of {mixed-integer} software. Computational experience is carried out on benchmark datasets from the UCI repository using a state-of-the-art off-the-shelf solver. Results show that the proposed model is effective in yielding a shallow interpretable tree approximating the tree-ensemble decision function.

翻译：模型的可解释性已成为机器学习中的关键问题，因为算法决策对现实世界应用的影响日益增长。随机森林或XgBoost等树集成方法是分类任务中的强大学习工具。然而，虽然组合多棵树可能比单棵树提供更高的预测质量，但它牺牲了可解释性，导致"黑箱"模型。鉴于此，我们旨在开发一种可解释的树集成模型表示方法，以深入理解其行为。首先，针对目标树集成模型，我们基于森林特征使用的热图表示开发了一种层级可视化工具，将特征的频率及其被选用的层级作为重要性指标。其次，我们提出了一种混合整数线性规划（MILP）公式，用于构建一个能够准确模仿目标模型预测的单一最优多变量树。目标是提供一种基于斜超平面分割的可解释替代模型，该模型仅使用根据定义的森林重要性指标确定的最相关特征。MILP模型包含基于特征在森林中出现频率的惩罚项，以进一步诱导分割的稀疏性。通过强化自然公式，我们提升了混合整数求解软件的计算性能。基于UCI基准数据集，采用最先进的开源求解器进行了计算实验。结果表明，所提模型能有效生成近似树集成决策函数的浅层可解释树。