We propose a novel machine learning method based on differentiable vortex particles to infer and predict fluid dynamics from a single video. The key design of our system is a particle-based latent space to encapsulate the hidden, Lagrangian vortical evolution underpinning the observable, Eulerian flow phenomena. We devise a novel differentiable vortex particle system in conjunction with their learnable, vortex-to-velocity dynamics mapping to effectively capture and represent the complex flow features in a reduced space. We further design an end-to-end training pipeline to directly learn and synthesize simulators from data, that can reliably deliver future video rollouts based on limited observation. The value of our method is twofold: first, our learned simulator enables the inference of hidden physics quantities (e.g. velocity field) purely from visual observation, to be used for motion analysis; secondly, it also supports future prediction, constructing the input video's sequel along with its future dynamics evolution. We demonstrate our method's efficacy by comparing quantitatively and qualitatively with a range of existing methods on both synthetic and real-world videos, displaying improved data correspondence, visual plausibility, and physical integrity.

翻译：我们提出了一种基于可微分涡旋粒子的新型机器学习方法，能够从单段视频中推断并预测流体动力学。该系统的核心设计在于构建基于粒子的潜空间，以封装可观测的欧拉流现象背后隐藏的拉格朗日涡旋演化规律。我们设计了一种新颖的可微分涡旋粒子系统，并配套开发了可学习的涡旋-速度动力学映射，从而在降维空间中有效捕获和表征复杂流场特征。进一步地，我们构建了端到端的训练流程，可直接从数据中学习并合成模拟器，基于有限观测可靠生成未来视频序列。本方法的价值体现在两个方面：其一，学习得到的模拟器可实现纯粹从视觉观测中推断隐藏物理量（如速度场），用于运动分析；其二，该模拟器支持未来预测，可构建输入视频的续集及其未来动力学演化。通过与现有多种方法在合成与真实视频上的定量和定性对比，我们验证了该方法在数据一致性、视觉可信度和物理完整性方面的优越性能。