Transformers have empowered many milestones across various fields and have recently been applied to solve partial differential equations (PDEs). However, since PDEs are typically discretized into large-scale meshes with complex geometries, it is challenging for Transformers to capture intricate physical correlations directly from massive individual points. Going beyond superficial and unwieldy meshes, we present Transolver based on a more foundational idea, which is learning intrinsic physical states hidden behind discretized geometries. Specifically, we propose a new Physics-Attention to adaptively split the discretized domain into a series of learnable slices of flexible shapes, where mesh points under similar physical states will be ascribed to the same slice. By calculating attention to physics-aware tokens encoded from slices, Transovler can effectively capture intricate physical correlations under complex geometrics, which also empowers the solver with endogenetic geometry-general modeling capacity and can be efficiently computed in linear complexity. Transolver achieves consistent state-of-the-art with 22% relative gain across six standard benchmarks and also excels in large-scale industrial simulations, including car and airfoil designs. Code is available at https://github.com/thuml/Transolver.

翻译：Transformer模型已在多个领域取得里程碑式突破，并开始应用于偏微分方程求解。然而，由于偏微分方程通常在复杂几何结构上被离散化为大规模网格，Transformer难以直接从海量离散点中捕捉复杂的物理关联。为突破表面化且笨重的网格表示局限，我们提出基于更本质思想的Transolver——其核心在于学习离散几何背后隐藏的固有物理状态。具体而言，我们设计了一种新型物理注意力机制，能够将离散域自适应地分割为一系列可学习的柔性形状切片，使处于相似物理状态的网格点归属于同一切片。通过对切片编码生成的物理感知令牌计算注意力，Transolver能有效捕捉复杂几何形态下的物理关联，这种机制同时赋予求解器内生的几何泛化建模能力，并可在线性复杂度内高效计算。在六项标准基准测试中，Transolver以22%的相对性能提升取得了一致的领先优势，并在汽车与翼型设计等大规模工业仿真任务中表现卓越。代码已开源：https://github.com/thuml/Transolver。