Equivariance encodes known symmetries into neural networks, often enhancing generalization. However, equivariant networks cannot break symmetries: the output of an equivariant network must, by definition, have at least the same self-symmetries as the input. This poses an important problem, both (1) for prediction tasks on domains where self-symmetries are common, and (2) for generative models, which must break symmetries in order to reconstruct from highly symmetric latent spaces. This fundamental limitation can be addressed by considering equivariant conditional distributions, instead of equivariant functions. We present novel theoretical results that establish necessary and sufficient conditions for representing such distributions. Concretely, this representation provides a practical framework for breaking symmetries in any equivariant network via randomized canonicalization. Our method, SymPE (Symmetry-breaking Positional Encodings), admits a simple interpretation in terms of positional encodings. This approach expands the representational power of equivariant networks while retaining the inductive bias of symmetry, which we justify through generalization bounds. Experimental results demonstrate that SymPE significantly improves performance of group-equivariant and graph neural networks across diffusion models for graphs, graph autoencoders, and lattice spin system modeling.

翻译：等变性将已知对称性编码到神经网络中，通常能增强泛化能力。然而，等变网络无法打破对称性：根据定义，等变网络的输出必须至少具有与输入相同的自对称性。这带来了一个重要问题，既（1）出现在自对称性普遍存在的领域中的预测任务，也（2）对于生成模型而言，它们必须打破对称性才能从高度对称的潜在空间中进行重构。这一根本性限制可以通过考虑等变条件分布而非等变函数来解决。我们提出了新的理论结果，确立了表示此类分布的充分必要条件。具体而言，该表示提供了一种通过随机化规范化在任何等变网络中打破对称性的实用框架。我们的方法SymPE（对称性破缺位置编码）可从位置编码的角度进行简单解释。该方法在保留对称性归纳偏置的同时扩展了等变网络的表示能力，我们通过泛化界限证明了这一点。实验结果表明，SymPE在图的扩散模型、图自编码器及晶格自旋系统建模中，显著提升了群等变网络与图神经网络的性能。