Many anatomical structures can be described by surface or volume meshes. Machine learning is a promising tool to extract information from these 3D models. However, high-fidelity meshes often contain hundreds of thousands of vertices, which creates unique challenges in building deep neural network architectures. Furthermore, patient-specific meshes may not be canonically aligned which limits the generalisation of machine learning algorithms. We propose LaB-GATr, a transfomer neural network with geometric tokenisation that can effectively learn with large-scale (bio-)medical surface and volume meshes through sequence compression and interpolation. Our method extends the recently proposed geometric algebra transformer (GATr) and thus respects all Euclidean symmetries, i.e. rotation, translation and reflection, effectively mitigating the problem of canonical alignment between patients. LaB-GATr achieves state-of-the-art results on three tasks in cardiovascular hemodynamics modelling and neurodevelopmental phenotype prediction, featuring meshes of up to 200,000 vertices. Our results demonstrate that LaB-GATr is a powerful architecture for learning with high-fidelity meshes which has the potential to enable interesting downstream applications. Our implementation is publicly available.

翻译：许多解剖结构可通过表面网格或体网格描述。机器学习是从这些三维模型中提取信息的有效工具。然而，高保真网格通常包含数十万个顶点，这为构建深度神经网络架构带来了独特挑战。此外，患者特定网格可能缺乏规范性对齐，限制了机器学习算法的泛化能力。我们提出LaB-GATr——一种具有几何标记化的变压器神经网络，通过序列压缩与插值技术，能够有效学习大规模（生物）医学表面与体网格。该方法扩展了近期提出的几何代数变压器（GATr），因此尊重所有欧几里得对称性（即旋转、平移和反射），有效缓解了患者间规范性对齐的问题。LaB-GATr在心血管血流动力学建模和神经发育表型预测的三项任务中取得了当前最优结果，涉及网格顶点数量高达20万。实验表明，LaB-GATr是面向高保真网格学习的高效架构，具备推动重要下游应用的潜力。我们的实现已公开发布。