Deep learning models have proven enormously successful at using multiple layers of representation to learn relevant features of structured data. Encoding physical symmetries into these models can improve performance on difficult tasks, and recent work has motivated the principle of parameter symmetry breaking and restoration as a unifying mechanism underlying their hierarchical learning dynamics. We evaluate the role of parameter symmetry and network expressivity in the generalisation behaviour of neural networks when learning a real-space renormalisation group (RG) transformation, using the central limit theorem (CLT) as a test case map. We consider simple multilayer perceptrons (MLPs) and graph neural networks (GNNs), and vary weight symmetries and activation functions across architectures. Our results reveal a competition between symmetry constraints and expressivity, with overly complex or overconstrained models generalising poorly. We analytically demonstrate this poor generalisation behaviour for certain constrained MLP architectures by recasting the CLT as a cumulant recursion relation and making use of an established framework to propagate cumulants through MLPs. We also empirically validate an extension of this framework from MLPs to GNNs, elucidating the internal information processing performed by these more complex models. These findings offer new insight into the learning dynamics of symmetric networks and their limitations in modelling structured physical transformations.

翻译：深度学习模型通过多层表示学习结构化数据的相关特征已取得巨大成功。将物理对称性编码到这些模型中可以提高困难任务的性能，近期研究提出了参数对称性破缺与恢复原理，作为其层次学习动态的统一机制。我们以中心极限定理（CLT）作为测试案例映射，评估了神经网络学习实空间重整化群（RG）变换时，参数对称性与网络表达能力在泛化行为中的作用。我们考察了简单多层感知机（MLP）和图神经网络（GNN），并在不同架构中调整权重对称性和激活函数。研究结果揭示了对称性约束与表达能力之间的竞争关系：过度复杂或过度受限的模型泛化能力较差。通过将CLT重构为累积量递推关系，并利用现有框架在MLP中传播累积量，我们解析地证明了某些受限MLP架构的泛化缺陷。我们还将该框架从MLP扩展到GNN并进行了实证验证，阐明了这些更复杂模型的内部信息处理机制。这些发现为对称网络的学习动态及其在建模结构化物理变换中的局限性提供了新的见解。