We present both a theoretical and a methodological framework that addresses a critical challenge in applying deep learning to physical systems: the reconciliation of non-linear expressiveness with SO(3)-equivariance in predictions of SO(3)-equivariant quantities, such as the electronic-structure Hamiltonian. Inspired by covariant theory in physics, we address this problem by exploring the mathematical relationships between SO(3)-invariant and SO(3)-equivariant quantities and their representations. We first construct theoretical SO(3)-invariant quantities derived from the SO(3)-equivariant regression targets, and use these invariant quantities as supervisory labels to guide the learning of high-quality SO(3)-invariant features. Given that SO(3)-invariance is preserved under non-linear operations, the encoding process for invariant features can extensively utilize non-linear mappings, thereby fully capturing the non-linear patterns inherent in physical systems. Building on this foundation, we propose a gradient-based mechanism to induce SO(3)-equivariant encodings of various degrees from the learned SO(3)-invariant features. This mechanism can incorporate non-linear expressive capabilities into SO(3)-equivariant representations, while theoretically preserving their equivariant properties as we prove. Our approach offers a promising general solution to the critical dilemma between equivariance and non-linear expressiveness in deep learning methodologies. We apply our theory and method to the electronic-structure Hamiltonian prediction tasks, demonstrating state-of-the-art performance across six benchmark databases.

翻译：我们提出了一个理论和方法论框架，解决了深度学习在物理系统应用中面临的关键挑战：在预测SO(3)等变量（如电子结构哈密顿量）时，如何协调非线性表达能力与SO(3)等变性之间的矛盾。受物理学中协变理论的启发，我们通过探索SO(3)不变与SO(3)等变量及其表示之间的数学关系来解决这一问题。我们首先构建了从SO(3)等变回归目标导出的理论SO(3)不变量，并利用这些不变量作为监督标签来引导学习高质量的SO(3)不变特征。由于SO(3)不变性在非线性操作下得以保持，不变特征的编码过程可以充分利用非线性映射，从而完整捕捉物理系统中固有的非线性模式。在此基础上，我们提出了一种基于梯度的机制，从学习到的SO(3)不变特征中诱导出不同阶次的SO(3)等变编码。该机制能够将非线性表达能力融入SO(3)等变表示，同时如我们所证明的，在理论上保留其等变性质。我们的方法为深度学习中等变性与非线性表达能力之间的关键困境提供了一种有前景的通用解决方案。我们将理论与方法应用于电子结构哈密顿量预测任务，在六个基准数据集上展示了最先进的性能。