We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schr\"odinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.

翻译：我们提出了一种使用自注意力机制的新型神经网络架构——波函数变换器（Psiformer），可作为求解多电子薛定谔方程的近似方法（或拟设），该方程是量子化学与材料科学的基础方程。该方程可通过第一性原理求解，无需外部训练数据。近年来，费米网络（FermiNet）和泡利网络（PauliNet）等深度神经网络已显著提升此类第一性原理计算的精度，但这些方法缺乏类似注意力机制来调控电子间的相互作用。研究表明，Psiformer可作为这些神经网络的直接替代方案，通常能大幅提升计算精度。尤其在较大分子体系中，基态能量可提升数十千卡/摩尔，实现相较此前方法的质变突破。这证明自注意力网络能够学习电子间复杂的量子力学关联，为在更大体系化学计算中达成前所未有的精度开辟了前景。