Methods for analyzing representations in neural systems are increasingly popular tools in neuroscience and mechanistic interpretability. Measures comparing neural activations across conditions, architectures, and species give scalable ways to understand information transformation within different neural networks. However, recent findings show that some metrics respond to spurious signals, leading to misleading results. Establishing benchmark test cases is thus essential for identifying the most reliable metric and potential improvements. We propose that compositional learning in recurrent neural networks (RNNs) can provide a test case for dynamical representation alignment metrics. Implementing this case allows us to evaluate if metrics can identify representations that develop throughout learning and determine if representations identified by metrics reflect the network's actual computations. Building both attractor and RNN based test cases, we show that the recently proposed Dynamical Similarity Analysis (DSA) is more noise robust and reliably identifies behaviorally relevant representations compared to prior metrics (Procrustes, CKA). We also demonstrate how such test cases can extend beyond metric evaluation to study new architectures. Specifically, testing DSA in modern (Mamba) state space models suggests that these models, unlike RNNs, may not require changes in recurrent dynamics due to their expressive hidden states. Overall, we develop test cases that showcase how DSA's enhanced ability to detect dynamical motifs makes it highly effective for identifying ongoing computations in RNNs and revealing how networks learn tasks.

翻译：分析神经系统中表征的方法在神经科学和机械可解释性领域日益流行。通过比较不同条件、架构和物种间神经激活的度量方法，为理解不同神经网络内的信息转换提供了可扩展的途径。然而，近期研究发现某些度量指标会对伪信号产生响应，导致误导性结果。因此，建立基准测试用例对于识别最可靠的度量方法及潜在改进至关重要。我们提出循环神经网络（RNNs）中的组合学习可为动态表征对齐度量提供测试案例。实施该案例使我们能够评估度量方法是否能识别在整个学习过程中发展的表征，并确定度量方法识别的表征是否反映了网络的实际计算。通过构建基于吸引子和RNN的测试案例，我们证明相较于先前的度量方法（Procrustes、CKA），近期提出的动态相似性分析（DSA）具有更强的噪声鲁棒性，并能可靠地识别与行为相关的表征。我们还展示了此类测试案例如何超越度量评估，用于研究新架构。具体而言，在现代（Mamba）状态空间模型中测试DSA表明，与RNN不同，这些模型可能因其表达性隐藏状态而无需改变循环动态。总体而言，我们开发的测试案例展示了DSA检测动态基元的增强能力如何使其在识别RNN中持续进行的计算及揭示网络如何学习任务方面极为有效。