Predicting physical dynamics from raw visual data remains a major challenge in AI. While recent video generation models have achieved impressive visual quality, they still cannot consistently generate physically plausible videos due to a lack of modeling of physical laws. Recent approaches combining 3D Gaussian splatting and physics engines can produce physically plausible videos, but are hindered by high computational costs in both reconstruction and simulation, and often lack robustness in complex real-world scenarios. To address these issues, we introduce Neural Gaussian Force Field (NGFF), an end-to-end neural framework that integrates 3D Gaussian perception with physics-based dynamic modeling to generate interactive, physically realistic 4D videos from multi-view RGB inputs, achieving two orders of magnitude faster than prior Gaussian simulators. To support training, we also present GSCollision, a 4D Gaussian dataset featuring diverse materials, multi-object interactions, and complex scenes, totaling over 640k rendered physical videos (~4 TB). Evaluations on synthetic and real 3D scenarios show NGFF's strong generalization and robustness in physical reasoning, advancing video prediction towards physics-grounded world models.

翻译：从原始视觉数据预测物理动力学仍然是人工智能领域的主要挑战。尽管近期视频生成模型已取得令人印象深刻的视觉质量，但由于缺乏物理定律建模，它们仍无法持续生成物理上合理的视频。结合3D高斯泼溅与物理引擎的最新方法虽能产生物理合理的视频，但在重建和仿真过程中均受限于高计算成本，且在复杂现实场景中常缺乏鲁棒性。为解决这些问题，我们提出了神经高斯力场（NGFF）——一种端到端的神经框架，它将3D高斯感知与基于物理的动态建模相结合，能够从多视角RGB输入生成交互式、物理真实的4维视频，其速度比现有高斯仿真器快两个数量级。为支持训练，我们还构建了GSCollision数据集，这是一个包含多种材料、多物体交互和复杂场景的4维高斯数据集，总计超过64万条渲染物理视频（约4TB）。在合成与真实3D场景上的评估表明，NGFF在物理推理方面具有强大的泛化能力和鲁棒性，推动了视频预测向物理约束的世界模型发展。