Using symmetry as an inductive bias in deep learning has been proven to be a principled approach for sample-efficient model design. However, the relationship between symmetry and the imperative for equivariance in neural networks is not always obvious. Here, we analyze a key limitation that arises in equivariant functions: their incapacity to break symmetry at the level of individual data samples. In response, we introduce a novel notion of 'relaxed equivariance' that circumvents this limitation. We further demonstrate how to incorporate this relaxation into equivariant multilayer perceptrons (E-MLPs), offering an alternative to the noise-injection method. The relevance of symmetry breaking is then discussed in various application domains: physics, graph representation learning, combinatorial optimization and equivariant decoding.

翻译：将对称性作为深度学习中的归纳偏置已被证明是实现样本高效模型设计的基本原则。然而，对称性与神经网络中等变要求的关联并非总是显而易见的。本文分析了等变函数的一个关键局限性：其在单个数据样本层面无法实现对称性破缺。为此，我们引入了一种名为"松弛等变性"的新概念以规避这一局限。进一步地，我们展示了如何将这种松弛机制融入等变多层感知器（E-MLPs），为噪声注入方法提供了替代方案。最后，我们讨论了对称性破缺在物理学、图表示学习、组合优化和等变解码等多个应用领域中的相关性。