Surrogate models approximate the solutions of partial differential equations (PDEs) for given geometries and boundary conditions. Conventional Transformer-based approaches typically employ a common feature-mixing mechanism across the solution domain, despite the presence of spatial variations at different scales. In this paper, we propose a mixture of transformer experts that combines a routing mechanism with multiscale supervision. Specifically, the individual experts are routed through a controller that utilizes learnable Laplacian matrices to compute weights, used to define a weighted sum of the experts for the output feature vectors at each point of the domain. Subsequently, a decoder receives the weighted sum of the experts output and predicts the solution fields. During training, the reference fields are decomposed into components at different spatial scales by repeated smoothing with a mesh Laplacian. We evaluate our approach on the official AirFRANS dataset using remeshed solution domains leading to unstructured triangular meshes of $4.9$k vertices. The experimental outcomes reveal lower mean relative $L_2$ error and interior mean squared errors among the compared models of this study. Code and data are publicly available in https://github.com/snousias/hodge-moe-pde
翻译:暂无翻译