This paper studies the problem of estimating physical properties (system identification) through visual observations. To facilitate geometry-aware guidance in physical property estimation, we introduce a novel hybrid framework that leverages 3D Gaussian representation to not only capture explicit shapes but also enable the simulated continuum to deduce implicit shapes during training. We propose a new dynamic 3D Gaussian framework based on motion factorization to recover the object as 3D Gaussian point sets across different time states. Furthermore, we develop a coarse-to-fine filling strategy to generate the density fields of the object from the Gaussian reconstruction, allowing for the extraction of object continuums along with their surfaces and the integration of Gaussian attributes into these continuums. In addition to the extracted object surfaces, the Gaussian-informed continuum also enables the rendering of object masks during simulations, serving as implicit shape guidance for physical property estimation. Extensive experimental evaluations demonstrate that our pipeline achieves state-of-the-art performance across multiple benchmarks and metrics. Additionally, we illustrate the effectiveness of the proposed method through real-world demonstrations, showcasing its practical utility. Our project page is at https://jukgei.github.io/project/gic.
翻译:本文研究通过视觉观测估计物理属性(系统辨识)的问题。为促进物理属性估计中的几何感知引导,我们引入一种新颖的混合框架,该框架利用3D高斯表示不仅捕捉显式几何形状,还能使模拟连续体在训练过程中推导隐式形状。我们提出一种基于运动分解的新型动态3D高斯框架,以跨不同时间状态将物体恢复为3D高斯点集。此外,我们开发了从粗到精的填充策略,从高斯重建生成物体的密度场,从而能够提取物体连续体及其表面,并将高斯属性集成到这些连续体中。除了提取的物体表面外,高斯信息化的连续体还能在仿真过程中渲染物体掩码,为物理属性估计提供隐式形状引导。大量实验评估表明,我们的流程在多个基准测试和指标上均实现了最先进的性能。此外,我们通过实际案例演示说明了所提方法的有效性,展示了其实用价值。项目页面位于 https://jukgei.github.io/project/gic。