As the complexity of machine learning (ML) models increases and the applications in different (and critical) domains grow, there is a strong demand for more interpretable and trustworthy ML. One straightforward and model-agnostic way to interpret complex ML models is to train surrogate models, such as rule sets and decision trees, that sufficiently approximate the original ones while being simpler and easier-to-explain. Yet, rule sets can become very lengthy, with many if-else statements, and decision tree depth grows rapidly when accurately emulating complex ML models. In such cases, both approaches can fail to meet their core goal, providing users with model interpretability. We tackle this by proposing DeforestVis, a visual analytics tool that offers user-friendly summarization of the behavior of complex ML models by providing surrogate decision stumps (one-level decision trees) generated with the adaptive boosting (AdaBoost) technique. Our solution helps users to explore the complexity vs fidelity trade-off by incrementally generating more stumps, creating attribute-based explanations with weighted stumps to justify decision making, and analyzing the impact of rule overriding on training instance allocation between one or more stumps. An independent test set allows users to monitor the effectiveness of manual rule changes and form hypotheses based on case-by-case investigations. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.

翻译：随着机器学习（ML）模型复杂性的提升及其在各类（关键）领域的应用日益广泛，对更具可解释性与可信赖性的ML模型的需求愈发迫切。训练替代模型（如规则集和决策树）是解释复杂ML模型的一种直接且与模型无关的方式——这类模型在更简单、更易解释的同时，能够充分逼近原始模型。然而，规则集可能包含大量if-else语句而变得冗长，决策树在精确模仿复杂ML模型时深度也会迅速增长。此时，这两种方法都可能无法实现其核心目标，即为用户提供模型可解释性。为此，我们提出DeforestVis这一可视化分析工具，通过提供基于自适应增强（AdaBoost）技术生成的替代决策桩（单层决策树），对复杂ML模型的行为进行用户友好的归纳。我们的解决方案通过逐步生成更多决策桩，帮助用户探索复杂度与保真度之间的权衡；借助加权决策桩构建基于属性的解释以辅助决策；并分析规则覆盖对训练实例在多个决策桩间分配的影响。独立的测试集使用户能够监控手动规则修改的有效性，并基于个案研究形成假设。我们通过两个使用案例及与数据分析师、模型开发者的专家访谈，展示了DeforestVis的适用性与实用性。