Recent developments in 3D vision have enabled successful progress in inferring neural fluid fields and realistic rendering of fluid dynamics. However, these methods require real-world flow captures, which demand dense video sequences and specialized lab setups, making the process costly and challenging. Scientific machine learning (SciML) foundation models, which are pretrained on extensive simulations of partial differential equations (PDEs), encode rich multiphysics knowledge and thus provide promising sources of domain priors for inferring fluid fields. Nevertheless, their potential to advance real-world vision problems remains largely underexplored, raising questions about the transferability and practical utility of these foundation models. In this work, we demonstrate that SciML foundation model can significantly improve the data efficiency of inferring real-world 3D fluid dynamics with improved generalization. At the core of our method is leveraging the strong forecasting capabilities and meaningful representations of SciML foundation models. We equip neural fluid fields with a novel collaborative training approach that utilizes augmented views and fluid features extracted by our foundation model. Our method demonstrates significant improvements in both quantitative metrics and visual quality, showcasing the practical applicability of SciML foundation models in real-world fluid dynamics.

翻译：近年来，三维视觉领域的发展在推断神经流体场和实现流体动力学的逼真渲染方面取得了显著进展。然而，这些方法需要真实世界的流体捕捉数据，这依赖于密集的视频序列和专门的实验室设置，导致过程成本高昂且具有挑战性。科学机器学习（SciML）基础模型在大量偏微分方程（PDEs）模拟数据上进行预训练，编码了丰富的多物理场知识，因此为推断流体场提供了有前景的领域先验来源。尽管如此，这些基础模型在推动真实世界视觉问题解决方面的潜力仍未得到充分探索，引发了关于其可迁移性和实际效用的疑问。在本工作中，我们证明了SciML基础模型能够显著提高推断真实世界三维流体动力学的数据效率，并改善其泛化能力。我们方法的核心在于利用SciML基础模型的强大预测能力和有意义的表征。我们为神经流体场配备了一种新颖的协同训练方法，该方法利用增强视图和由我们基础模型提取的流体特征。我们的方法在定量指标和视觉质量上均展现出显著提升，彰显了SciML基础模型在真实世界流体动力学中的实际应用价值。