The current approach to ML model design is either to choose a flexible Blackbox model and explain it post hoc or to start with an interpretable model. Blackbox models are flexible but difficult to explain, whereas interpretable models are designed to be explainable. However, developing interpretable models necessitates extensive ML knowledge, and the resulting models tend to be less flexible, offering potentially subpar performance compared to their Blackbox equivalents. This paper aims to blur the distinction between a post hoc explanation of a BlackBox and constructing interpretable models. We propose beginning with a flexible BlackBox model and gradually \emph{carving out} a mixture of interpretable models and a \emph{residual network}. Our design identifies a subset of samples and \emph{routes} them through the interpretable models. The remaining samples are routed through a flexible residual network. We adopt First Order Logic (FOL) as the interpretable model's backbone, which provides basic reasoning on concepts retrieved from the BlackBox model. On the residual network, we repeat the method until the proportion of data explained by the residual network falls below a desired threshold. Our approach offers several advantages. First, the mixture of interpretable and flexible residual networks results in almost no compromise in performance. Second, the route, interpret, and repeat approach yields a highly flexible interpretable model. Our extensive experiment demonstrates the performance of the model on various datasets. We show that by editing the FOL model, we can fix the shortcut learned by the original BlackBox model. Finally, our method provides a framework for a hybrid symbolic-connectionist network that is simple to train and adaptable to many applications.

翻译：当前机器学习模型设计的常见做法是：要么选择灵活的“黑箱”模型并事后对其进行解释，要么从一开始就采用可解释模型。黑箱模型灵活性高但难以解释，而可解释模型则天生具备可解释性。然而，开发可解释模型需要深厚的机器学习知识，且所得模型通常灵活性较差，性能可能逊于同等规模的黑箱模型。本文旨在模糊事后解释黑箱模型与构建可解释模型之间的界限。我们提出从灵活的黑箱模型出发，逐步“雕刻”出可解释模型与“残差网络”的混合体。我们的设计首先识别出一个样本子集，并将这些样本“路由”至可解释模型进行处理，其余样本则被路由至灵活的残差网络。我们采用一阶逻辑作为可解释模型的核心框架，它能基于从黑箱模型中提取的概念进行基础推理。对于残差网络，我们重复上述过程，直至残差网络所解释的数据比例降至预设阈值以下。该方法具有多项优势：首先，可解释模型与灵活残差网络的混合几乎不牺牲性能；其次，“路由、解释、重复”的策略生成了高度灵活的可解释模型。通过大量实验，我们验证了该模型在多种数据集上的性能表现。研究表明，通过编辑一阶逻辑模型，我们能够修正原始黑箱模型习得的捷径。最后，我们的方法为构建易于训练且适用于多种应用的混合符号-联结主义网络提供了框架。