As the complexity of machine learning (ML) models increases and their application in different (and critical) domains grows, there is a strong demand for more interpretable and trustworthy ML. A direct, model-agnostic, way to interpret such models is to train surrogate models, such as rule sets and decision trees, that sufficiently approximate the original ones while being simpler and easier-to-explain. Yet, rule sets can become very lengthy, with many if-else statements, and decision tree depth grows rapidly when accurately emulating complex ML models. In such cases, both approaches can fail to meet their core goal, providing users with model interpretability. To tackle this, we propose DeforestVis, a visual analytics tool that offers user-friendly summarization of the behavior of complex ML models by providing surrogate decision stumps (one-level decision trees) generated with the adaptive boosting (AdaBoost) technique. DeforestVis helps users to explore the complexity vs fidelity trade-off by incrementally generating more stumps, creating attribute-based explanations with weighted stumps to justify decision making, and analyzing the impact of rule overriding on training instance allocation between one or more stumps. An independent test set allows users to monitor the effectiveness of manual rule changes and form hypotheses based on case-by-case analyses. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.

翻译：随着机器学习模型复杂度的不断提升及其在关键领域的广泛应用，对更具可解释性和可信赖性的机器学习需求日益迫切。一种直接的、模型无关的解释方式是通过训练替代模型（如规则集和决策树）来充分逼近原始模型，同时保持其更简单、更易解释的特性。然而，当精确模拟复杂机器学习模型时，规则集可能因包含大量if-else语句而变得冗长，决策树深度也会急剧增长。这两种方法在此类情况下均难以实现其核心目标——为用户提供模型可解释性。为此，我们提出DeforestVis——一种可视化分析工具，通过自适应提升（AdaBoost）技术生成替代决策桩（单层决策树），以用户友好的方式总结复杂机器学习模型的行为。DeforestVis通过增量生成更多决策桩、利用加权决策桩创建基于属性的解释以辅助决策分析，以及检验规则覆盖对训练实例在多个决策桩间分配的影响，帮助用户探索复杂度与保真度之间的权衡。独立测试集使用户能够监控手工规则修改的有效性，并基于个案分析形成假设。通过两个用例及与数据分析师和模型开发者的专家访谈，我们验证了DeforestVis的适用性与实用性。