Molecular modeling, a central topic in quantum mechanics, aims to accurately calculate the properties and simulate the behaviors of molecular systems. The molecular model is governed by physical laws, which impose geometric constraints such as invariance and equivariance to coordinate rotation and translation. While numerous deep learning approaches have been developed to learn molecular representations under these constraints, most of them are built upon heuristic and costly modules. We argue that there is a strong need for a general and flexible framework for learning both invariant and equivariant features. In this work, we introduce a novel Transformer-based molecular model called GeoMFormer to achieve this goal. Using the standard Transformer modules, two separate streams are developed to maintain and learn invariant and equivariant representations. Carefully designed cross-attention modules bridge the two streams, allowing information fusion and enhancing geometric modeling in each stream. As a general and flexible architecture, we show that many previous architectures can be viewed as special instantiations of GeoMFormer. Extensive experiments are conducted to demonstrate the power of GeoMFormer. All empirical results show that GeoMFormer achieves strong performance on both invariant and equivariant tasks of different types and scales. Code and models will be made publicly available at https://github.com/c-tl/GeoMFormer.

翻译：分子建模作为量子力学的核心课题，旨在精确计算分子系统的性质并模拟其行为。分子模型受物理定律支配，这些定律施加了几何约束，例如对坐标旋转和平移的不变性及等变性。尽管目前已开发出众多深度学习方法来在这些约束下学习分子表示，但大多数方法都建立在启发式且计算代价高昂的模块之上。我们认为，亟需一个通用且灵活的框架来同时学习不变特征和等变特征。在本工作中，我们引入了一种名为GeoMFormer的新型基于Transformer的分子模型以实现此目标。利用标准Transformer模块，我们开发了两个独立的流来维持和学习不变表示与等变表示。精心设计的交叉注意力模块桥接了这两个流，实现了信息融合并增强了各自流中的几何建模能力。作为一个通用且灵活的架构，我们证明了先前的许多架构可被视为GeoMFormer的特例。我们进行了大量实验以验证GeoMFormer的能力。所有实证结果表明，GeoMFormer在不同类型和规模的不变性与等变性任务上均取得了优异的性能。代码和模型将在 https://github.com/c-tl/GeoMFormer 公开提供。