A multitude of (dis)similarity measures between neural network representations have been proposed, resulting in a fragmented research landscape. Most of these measures fall into one of two categories. First, measures such as linear regression, canonical correlations analysis (CCA), and shape distances, all learn explicit mappings between neural units to quantify similarity while accounting for expected invariances. Second, measures such as representational similarity analysis (RSA), centered kernel alignment (CKA), and normalized Bures similarity (NBS) all quantify similarity in summary statistics, such as stimulus-by-stimulus kernel matrices, which are already invariant to expected symmetries. Here, we take steps towards unifying these two broad categories of methods by observing that the cosine of the Riemannian shape distance (from category 1) is equal to NBS (from category 2). We explore how this connection leads to new interpretations of shape distances and NBS, and draw contrasts of these measures with CKA, a popular similarity measure in the deep learning literature.

翻译：已有大量度量神经网络表征间（相异）相似性的方法被提出，导致研究格局碎片化。这些度量大多可归为两类：第一类方法（如线性回归、典型相关分析（CCA）和形状距离）通过学习神经单元间的显式映射，在考虑预期不变性的同时量化相似性；第二类方法（如表征相似性分析（RSA）、中心核对齐（CKA）和归一化Bures相似性（NBS））则通过汇总统计量（如逐刺激核矩阵）直接量化已具有预期对称不变性的相似性。本文观察到黎曼形状距离（第一类）的余弦值等于NBS（第二类），从而朝着统一这两大类方法迈出关键一步。我们探究了这一联系如何催生对形状距离和NBS的新解释，并将这些度量与深度学习文献中广受欢迎的相似性度量CKA进行了对比分析。