Existing approaches to system identification (estimating the physical parameters of an object) from videos assume known object geometries. This precludes their applicability in a vast majority of scenes where object geometries are complex or unknown. In this work, we aim to identify parameters characterizing a physical system from a set of multi-view videos without any assumption on object geometry or topology. To this end, we propose "Physics Augmented Continuum Neural Radiance Fields" (PAC-NeRF), to estimate both the unknown geometry and physical parameters of highly dynamic objects from multi-view videos. We design PAC-NeRF to only ever produce physically plausible states by enforcing the neural radiance field to follow the conservation laws of continuum mechanics. For this, we design a hybrid Eulerian-Lagrangian representation of the neural radiance field, i.e., we use the Eulerian grid representation for NeRF density and color fields, while advecting the neural radiance fields via Lagrangian particles. This hybrid Eulerian-Lagrangian representation seamlessly blends efficient neural rendering with the material point method (MPM) for robust differentiable physics simulation. We validate the effectiveness of our proposed framework on geometry and physical parameter estimation over a vast range of materials, including elastic bodies, plasticine, sand, Newtonian and non-Newtonian fluids, and demonstrate significant performance gain on most tasks.

翻译：现有的基于视频的系统辨识（估计物体物理参数）方法均假设目标几何形状已知，这限制了其在绝大多数物体几何复杂或未知场景中的适用性。本研究旨在通过多视角视频集描述物理系统的参数，无需对物体几何或拓扑结构作任何假设。为此，我们提出“物理增强连续神经辐射场”（PAC-NeRF），用于从多视角视频中同时估计高动态物体的未知几何与物理参数。通过强制神经辐射场遵循连续介质力学守恒定律，我们设计PAC-NeRF仅生成物理可行的状态。具体而言，我们构建了神经辐射场的混合欧拉-拉格朗日表征：利用欧拉网格表征NeRF的密度场与颜色场，同时通过拉格朗日粒子平流驱动神经辐射场。这种混合表征无缝融合了高效神经渲染与用于鲁棒可微物理仿真的物质点法（MPM）。我们在涵盖弹性体、橡皮泥、沙土、牛顿及非牛顿流体等广泛材料的几何与物理参数估计任务上验证了本框架的有效性，并在多数任务中展现出显著性能提升。