Using symmetry as an inductive bias in deep learning has been proven to be a principled approach for sample-efficient model design. However, the relationship between symmetry and the imperative for equivariance in neural networks is not always obvious. Here, we analyze a key limitation that arises in equivariant functions: their incapacity to break symmetry at the level of individual data samples. In response, we introduce a novel notion of 'relaxed equivariance' that circumvents this limitation. We further demonstrate how to incorporate this relaxation into equivariant multilayer perceptrons (E-MLPs), offering an alternative to the noise-injection method. The relevance of symmetry breaking is then discussed in various application domains: physics, graph representation learning, combinatorial optimization and equivariant decoding.

翻译：利用对称性作为深度学习中的归纳偏置已被证明是样本高效模型设计的根本方法。然而，对称性与神经网络中等变性要求之间的关系并不总是显而易见的。在此，我们分析了等变函数面临的一个关键限制：它们在单个数据样本层面无法打破对称性。针对这一问题，我们引入了一种新的"松弛等变"概念，有效规避了这一限制。我们进一步展示了如何将这种松弛方法融入等变多层感知机（E-MLP），为噪声注入方法提供了替代方案。最后，我们讨论了对称性破缺在物理、图表示学习、组合优化和等变解码等多个应用领域中的相关性。