As the complexity of machine learning (ML) models increases and their application in different (and critical) domains grows, there is a strong demand for more interpretable and trustworthy ML. A direct, model-agnostic, way to interpret such models is to train surrogate models-such as rule sets and decision trees-that sufficiently approximate the original ones while being simpler and easier-to-explain. Yet, rule sets can become very lengthy, with many if-else statements, and decision tree depth grows rapidly when accurately emulating complex ML models. In such cases, both approaches can fail to meet their core goal-providing users with model interpretability. To tackle this, we propose DeforestVis, a visual analytics tool that offers summarization of the behaviour of complex ML models by providing surrogate decision stumps (one-level decision trees) generated with the Adaptive Boosting (AdaBoost) technique. DeforestVis helps users to explore the complexity versus fidelity trade-off by incrementally generating more stumps, creating attribute-based explanations with weighted stumps to justify decision making, and analysing the impact of rule overriding on training instance allocation between one or more stumps. An independent test set allows users to monitor the effectiveness of manual rule changes and form hypotheses based on case-by-case analyses. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.

翻译：随着机器学习模型复杂度的增加及其在不同（关键）领域中的应用日益广泛，对更具可解释性和可信赖性的机器学习需求愈发迫切。一种直接的、模型无关的解释方法是通过训练替代模型（如规则集和决策树）来充分近似原始模型，同时保持模型更简单且更易于解释。然而，当准确模拟复杂机器学习模型时，规则集可能包含大量if-else语句而变得冗长，决策树深度也会迅速增长。在此类情况下，这两种方法均可能无法实现其核心目标——为用户提供模型可解释性。为此，我们提出DeforestVis，一种可视化分析工具，通过自适应增强技术生成的替代决策桩（单层决策树）对复杂机器学习模型的行为进行总结。DeforestVis通过逐步生成更多决策桩，帮助用户探索复杂度与保真度之间的权衡；利用加权决策桩创建基于属性的解释以验证决策合理性；并分析规则覆盖对训练实例在多个决策桩间分配的影响。独立测试集允许用户监控人工规则修改的有效性，并基于个案分析形成假设。通过两个用例及与数据分析师和模型开发者的专家访谈，我们展示了DeforestVis的适用性与实用性。