The self-attention mechanism has been adopted in several widely-used message-passing neural networks (MPNNs) (e.g., GATs), which adaptively controls the amount of information that flows along the edges of the underlying graph. This usage of attention has made such models a baseline for studies on explainable AI (XAI) since interpretations via attention have been popularized in various domains (e.g., natural language processing and computer vision). However, existing studies often use naive calculations to derive attribution scores from attention, and do not take the precise and careful calculation of edge attribution into consideration. In our study, we aim to fill the gap between the widespread usage of attention-enabled MPNNs and their potential in largely under-explored explainability, a topic that has been actively investigated in other areas. To this end, as the first attempt, we formalize the problem of edge attribution from attention weights in GNNs. Then, we propose GATT, an edge attribution calculation method built upon the computation tree. Through comprehensive experiments, we demonstrate the effectiveness of our proposed method when evaluating attributions from GATs. Conversely, we empirically validate that simply averaging attention weights over graph attention layers is insufficient to interpret the GAT model's behavior. Code is publicly available at https://github.com/jordan7186/GAtt/tree/main.

翻译：自注意力机制已被多种广泛使用的消息传递神经网络（MPNNs）（例如GATs）所采用，其自适应地控制沿底层图边流动的信息量。这种注意力的应用使得此类模型成为可解释人工智能（XAI）研究的基线，因为通过注意力进行解释的方法已在多个领域（例如自然语言处理和计算机视觉）得到普及。然而，现有研究通常采用简单计算从注意力中推导归因分数，并未充分考虑边归因的精确细致计算。在本研究中，我们旨在填补启用注意力的MPNNs的广泛使用与其在尚未充分探索的可解释性方面的潜力之间的差距——这一主题在其他领域已被积极研究。为此，我们首次尝试形式化了图神经网络中基于注意力权重的边归因问题。随后，我们提出了GATT——一种基于计算树构建的边归因计算方法。通过全面实验，我们证明了所提方法在评估GATs生成的归因时的有效性。相反地，我们通过实证验证了简单平均图注意力层间的注意力权重不足以解释GAT模型的行为。代码公开于https://github.com/jordan7186/GAtt/tree/main。