Deep models have achieved impressive progress in solving partial differential equations (PDEs). A burgeoning paradigm is learning neural operators to approximate the input-output mappings of PDEs. While previous deep models have explored the multiscale architectures and various operator designs, they are limited to learning the operators as a whole in the coordinate space. In real physical science problems, PDEs are complex coupled equations with numerical solvers relying on discretization into high-dimensional coordinate space, which cannot be precisely approximated by a single operator nor efficiently learned due to the curse of dimensionality. We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Going beyond the coordinate space, LSM enables an attention-based hierarchical projection network to reduce the high-dimensional data into a compact latent space in linear time. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space that approximates complex input-output mappings via learning multiple basis operators, enjoying nice theoretical guarantees for convergence and approximation. Experimentally, LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks covering both solid and fluid physics. Code is available at https://github.com/thuml/Latent-Spectral-Models.

翻译：深度模型在求解偏微分方程方面取得了显著进展。一个新兴范式是学习神经算子以逼近偏微分方程的输入-输出映射。尽管以往的深度模型探索了多尺度架构和多种算子设计，但它们仍局限于在坐标空间中以整体方式学习算子。在实际物理科学问题中，偏微分方程是复杂的耦合方程，其数值求解器依赖于离散化处理进入高维坐标空间，这使得它们既无法通过单一算子精确逼近，也难以规避维度灾难实现高效学习。我们提出隐式谱模型（LSM），旨在成为高维偏微分方程的高效精确求解器。LSM突破坐标空间限制，采用基于注意力的层次化投影网络，能在线性时间内将高维数据压缩至紧致隐空间。受数值分析中经典谱方法启发，我们在隐空间中设计神经谱块以求解偏微分方程——通过学习多个基算子来逼近复杂输入-输出映射，并具备收敛性与逼近性的良好理论保证。实验表明，LSM在涵盖固体与流体物理的七个基准测试中均达到一致的最优性能，平均相对提升率达11.5%。代码开源至https://github.com/thuml/Latent-Spectral-Models。