Equivariance is a powerful inductive bias in neural networks, improving generalisation and physical consistency. Recently, however, non-equivariant models have regained attention, due to their better runtime performance and imperfect symmetries that might arise in real-world applications. This has motivated the development of approximately equivariant models that strike a middle ground between respecting symmetries and fitting the data distribution. Existing approaches in this field usually apply sample-based regularisers which depend on data augmentation at training time, incurring a high sample complexity, in particular for continuous groups such as $SO(3)$. This work instead approaches approximate equivariance via a projection-based regulariser which leverages the orthogonal decomposition of linear layers into equivariant and non-equivariant components. In contrast to existing methods, this penalises non-equivariance at an operator level across the full group orbit, rather than point-wise. We present a mathematical framework for computing the non-equivariance penalty exactly and efficiently in both the spatial and spectral domain. In our experiments, our method consistently outperforms prior approximate equivariance approaches in both model performance and efficiency, achieving substantial runtime gains over sample-based regularisers.

翻译：等变性是神经网络中一种强大的归纳偏置，能够提升模型的泛化能力与物理一致性。然而近期，非等变模型因其更优的运行效率以及现实应用中可能存在的非完美对称性而重新受到关注。这推动了近似等变模型的发展，旨在对称性约束与数据分布拟合之间寻求平衡。该领域的现有方法通常采用基于样本的正则化器，其依赖于训练时的数据增强，导致较高的样本复杂度，尤其对于连续群（如 $SO(3)$）而言。本研究则通过一种基于投影的正则化器实现近似等变性，该正则化器利用了线性层向等变与非等变分量的正交分解。与现有方法不同，本方法在全群轨道上以算子层面惩罚非等变性，而非逐点惩罚。我们提出了一个数学框架，可在空间域与谱域中精确且高效地计算非等变惩罚项。实验结果表明，我们的方法在模型性能与效率上均持续优于先前的近似等变方法，相较于基于样本的正则化器实现了显著的运行时间增益。