We propose a causal interpretation of self-attention in the Transformer neural network architecture. We interpret self-attention as a mechanism that estimates a structural equation model for a given input sequence of symbols (tokens). The structural equation model can be interpreted, in turn, as a causal structure over the input symbols under the specific context of the input sequence. Importantly, this interpretation remains valid in the presence of latent confounders. Following this interpretation, we estimate conditional independence relations between input symbols by calculating partial correlations between their corresponding representations in the deepest attention layer. This enables learning the causal structure over an input sequence using existing constraint-based algorithms. In this sense, existing pre-trained Transformers can be utilized for zero-shot causal-discovery. We demonstrate this method by providing causal explanations for the outcomes of Transformers in two tasks: sentiment classification (NLP) and recommendation.

翻译：我们提出了一种对Transformer神经网络架构中自注意力机制的因果解释。我们将自注意力机制视为对给定输入符号（令牌）序列估计结构方程模型的方法。这一结构方程模型可进一步解释为输入序列特定语境下输入符号间的因果结构。重要的是，该解释在存在潜在混杂因素时仍然有效。遵循这一解释，我们通过计算最深注意力层中对应表示的偏相关系数，来估计输入符号间的条件独立关系，从而利用现有基于约束的算法学习输入序列上的因果结构。在此意义上，现有预训练Transformer可用于零样本因果发现。我们通过两个任务（情感分类（自然语言处理）与推荐系统）中Transformer输出结果的因果解释，演示了该方法。