Symmetry is a fundamental tool in the exploration of a broad range of complex systems. In machine learning symmetry has been explored in both models and data. In this paper we seek to connect the symmetries arising from the architecture of a family of models with the symmetries of that family's internal representation of data. We do this by calculating a set of fundamental symmetry groups, which we call the intertwiner groups of the model. We connect intertwiner groups to a model's internal representations of data through a range of experiments that probe similarities between hidden states across models with the same architecture. Our work suggests that the symmetries of a network are propagated into the symmetries in that network's representation of data, providing us with a better understanding of how architecture affects the learning and prediction process. Finally, we speculate that for ReLU networks, the intertwiner groups may provide a justification for the common practice of concentrating model interpretability exploration on the activation basis in hidden layers rather than arbitrary linear combinations thereof.

翻译：对称性是探索广泛复杂系统的基本工具。在机器学习中，对称性已在模型与数据两个层面得到研究。本文旨在建立模型架构所引发的对称性与该家族模型数据内部表示对称性之间的关联。我们通过计算一组基本对称群（称之为模型的交织子群）来实现这一目标。通过一系列实验——这些实验比较具有相同架构的模型间隐藏状态的相似性——我们将交织子群与模型的数据内部表示联系起来。研究表明，网络的对称性会传递至该网络数据表示的对称性中，从而加深我们对架构如何影响学习与预测过程的理解。最后，我们推测对于ReLU网络而言，交织子群可能为"将模型可解释性探索集中在隐藏层激活基（而非其任意线性组合）上"这一常见实践提供理论依据。