Causal abstraction is a promising theoretical framework for explainable artificial intelligence that defines when an interpretable high-level causal model is a faithful simplification of a low-level deep learning system. However, existing causal abstraction methods have two major limitations: they require a brute-force search over alignments between the high-level model and the low-level one, and they presuppose that variables in the high-level model will align with disjoint sets of neurons in the low-level one. In this paper, we present distributed alignment search (DAS), which overcomes these limitations. In DAS, we find the alignment between high-level and low-level models using gradient descent rather than conducting a brute-force search, and we allow individual neurons to play multiple distinct roles by analyzing representations in non-standard bases-distributed representations. Our experiments show that DAS can discover internal structure that prior approaches miss. Overall, DAS removes previous obstacles to conducting causal abstraction analyses and allows us to find conceptual structure in trained neural nets.

翻译：因果抽象是可解释人工智能领域一种具有前景的理论框架，它定义了可解释的高层因果模型何时能够忠实地简化低层深度学习系统。然而，现有的因果抽象方法存在两个主要局限：它们需要对高层模型与低层模型之间的对齐进行暴力搜索，并且预设高层模型中的变量将与低层模型中的不相交神经元集合对齐。在本文中，我们提出分布式对齐搜索（DAS），克服了这些局限。在DAS中，我们利用梯度下降而非暴力搜索来寻找高层与低层模型之间的对齐，并通过分析非标准基（分布式表征）中的表示，允许单个神经元扮演多个不同角色。实验表明，DAS能够发现先前方法遗漏的内部结构。总体而言，DAS消除了开展因果抽象分析的既有障碍，使我们得以在训练后的神经网络中发现概念结构。