Reconstructing and simulating elastic objects from visual observations is crucial for applications in computer vision and robotics. Existing methods, such as 3D Gaussians, provide modeling for 3D appearance and geometry but lack the ability to simulate physical properties or optimize parameters for heterogeneous objects. We propose Spring-Gaus, a novel framework that integrates 3D Gaussians with physics-based simulation for reconstructing and simulating elastic objects from multi-view videos. Our method utilizes a 3D Spring-Mass model, enabling the optimization of physical parameters at the individual point level while decoupling the learning of physics and appearance. This approach achieves great sample efficiency, enhances generalization, and reduces sensitivity to the distribution of simulation particles. We evaluate Spring-Gaus on both synthetic and real-world datasets, demonstrating accurate reconstruction and simulation of elastic objects. This includes future prediction and simulation under varying initial states and environmental parameters. Project page: https://zlicheng.com/spring_gaus.

翻译：从视觉观测中重建并仿真弹性物体，对于计算机视觉与机器人领域的应用至关重要。现有方法（如三维高斯）虽能建模三维外观与几何结构，但缺乏模拟物理属性或针对异质物体优化参数的能力。我们提出Spring-Gaus——一种将三维高斯与基于物理的仿真相结合的新型框架，旨在从多视角视频中重建并仿真弹性物体。该方法采用三维弹簧-质量模型，可在解耦物理学习与外观学习的同时，实现逐点的物理参数优化。该框架具备出色的样本效率、增强的泛化能力，并降低了对仿真粒子分布的敏感性。我们在合成数据集与真实数据集上对Spring-Gaus进行评估，验证了其在弹性物体重建与仿真中的准确性，包括在不同初始状态与环境参数下的未来预测与仿真能力。项目页面：https://zlicheng.com/spring_gaus