ML model design either starts with an interpretable model or a Blackbox and explains it post hoc. Blackbox models are flexible but difficult to explain, while interpretable models are inherently explainable. Yet, interpretable models require extensive ML knowledge and tend to be less flexible and underperforming than their Blackbox variants. This paper aims to blur the distinction between a post hoc explanation of a Blackbox and constructing interpretable models. Beginning with a Blackbox, we iteratively carve out a mixture of interpretable experts (MoIE) and a residual network. Each interpretable model specializes in a subset of samples and explains them using First Order Logic (FOL), providing basic reasoning on concepts from the Blackbox. We route the remaining samples through a flexible residual. We repeat the method on the residual network until all the interpretable models explain the desired proportion of data. Our extensive experiments show that our route, interpret, and repeat approach (1) identifies a diverse set of instance-specific concepts with high concept completeness via MoIE without compromising in performance, (2) identifies the relatively harder samples to explain via residuals, (3) outperforms the interpretable by-design models by significant margins during test-time interventions, and (4) fixes the shortcut learned by the original Blackbox. The code for MoIE is publicly available at: https://github.com/batmanlab/ICML-2023-Route-interpret-repeat.

翻译：机器学习模型设计通常从可解释模型或后验解释的黑盒开始。黑盒模型灵活但难以解释，而可解释模型具有内在可解释性。然而，可解释模型需要大量的机器学习知识，且其性能往往不如黑盒变体灵活且表现不佳。本文旨在模糊黑盒后验解释与构建可解释模型之间的界限。从黑盒出发，我们迭代地剥离出可解释专家混合体（MoIE）和残差网络。每个可解释模型专注于子样本集，并使用一阶逻辑（FOL）进行解释，提供基于黑盒概念的基本推理。我们通过灵活的残差网络路由剩余样本。对残差网络重复上述方法，直到所有可解释模型覆盖所需的数据比例。大量实验表明，我们的路由-解释-重复方法能够：（1）在不牺牲性能的前提下，通过MoIE识别出具有高概念完备性的多样化实例特定概念；（2）通过残差网络识别出相对难以解释的样本；（3）在测试时干预中显著优于设计上的可解释模型；（4）修复原始黑盒学习到的捷径。MoIE的代码已公开：https://github.com/batmanlab/ICML-2023-Route-interpret-repeat.