As the complexity of machine learning (ML) models increases and the applications in different (and critical) domains grow, there is a strong demand for more interpretable and trustworthy ML. One straightforward and model-agnostic way to interpret complex ML models is to train surrogate models, such as rule sets and decision trees, that sufficiently approximate the original ones while being simpler and easier-to-explain. Yet, rule sets can become very lengthy, with many if-else statements, and decision tree depth grows rapidly when accurately emulating complex ML models. In such cases, both approaches can fail to meet their core goal, providing users with model interpretability. We tackle this by proposing DeforestVis, a visual analytics tool that offers user-friendly summarization of the behavior of complex ML models by providing surrogate decision stumps (one-level decision trees) generated with the adaptive boosting (AdaBoost) technique. Our solution helps users to explore the complexity vs fidelity trade-off by incrementally generating more stumps, creating attribute-based explanations with weighted stumps to justify decision making, and analyzing the impact of rule overriding on training instance allocation between one or more stumps. An independent test set allows users to monitor the effectiveness of manual rule changes and form hypotheses based on case-by-case investigations. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.

翻译：随着机器学习模型复杂度的提升及其在不同（关键）领域应用的扩展，对更可解释、更可信的机器学习的需求日益迫切。一种直观且与模型无关的解释复杂机器学习模型的方法，是训练替代模型（如规则集和决策树），这些模型在充分逼近原始模型的同时更简单且更易解释。然而，当准确模拟复杂机器学习模型时，规则集可能因包含大量if-else语句而变得冗长，决策树深度也会迅速增长。此时，这两种方法均可能无法实现其核心目标——为用户提供模型可解释性。为此，我们提出DeforestVis，一种可视化分析工具，通过提供基于自适应增强（AdaBoost）技术生成的替代决策桩（单层决策树），实现对复杂机器学习模型行为的用户友好型总结。我们的解决方案通过增量生成更多决策桩，帮助用户探索复杂度与保真度之间的权衡；通过加权决策桩创建基于属性的解释以阐明决策依据；并分析规则覆盖对训练实例在单个或多个决策桩间分配的影响。独立测试集允许用户监测手动规则修改的有效性，并基于个案研究形成假设。我们通过两个用例及与数据分析师和模型开发者的专家访谈，展示了DeforestVis的适用性和实用性。