Estimating physical properties for visual data is a crucial task in computer vision, graphics, and robotics, underpinning applications such as augmented reality, physical simulation, and robotic grasping. However, this area remains under-explored due to the inherent ambiguities in physical property estimation. To address these challenges, we introduce GaussianProperty, a training-free framework that assigns physical properties of materials to 3D Gaussians. Specifically, we integrate the segmentation capability of SAM with the recognition capability of GPT-4V(ision) to formulate a global-local physical property reasoning module for 2D images. Then we project the physical properties from multi-view 2D images to 3D Gaussians using a voting strategy. We demonstrate that 3D Gaussians with physical property annotations enable applications in physics-based dynamic simulation and robotic grasping. For physics-based dynamic simulation, we leverage the Material Point Method (MPM) for realistic dynamic simulation. For robot grasping, we develop a grasping force prediction strategy that estimates a safe force range required for object grasping based on the estimated physical properties. Extensive experiments on material segmentation, physics-based dynamic simulation, and robotic grasping validate the effectiveness of our proposed method, highlighting its crucial role in understanding physical properties from visual data. Online demo, code, more cases and annotated datasets are available on \href{https://Gaussian-Property.github.io}{this https URL}.

翻译：从视觉数据中估计物理属性是计算机视觉、图形学和机器人学领域的关键任务，为增强现实、物理仿真和机器人抓取等应用提供基础支撑。然而，由于物理属性估计固有的模糊性，该领域的研究仍显不足。为应对这些挑战，本文提出高斯属性——一种无需训练即可为3D高斯赋予材料物理属性的框架。具体而言，我们整合SAM的分割能力与GPT-4V(ision)的识别能力，构建面向二维图像的全局-局部物理属性推理模块。随后通过投票策略将多视角二维图像的物理属性映射至3D高斯表示。实验证明，带有物理属性标注的3D高斯能够支持基于物理的动态仿真与机器人抓取应用。在物理动态仿真方面，我们采用物质点法实现高真实度动态模拟；在机器人抓取方面，我们开发了基于估计物理属性的抓握力预测策略，可估算物体抓取所需的安全施力范围。在材料分割、物理动态仿真和机器人抓取任务上的大量实验验证了所提方法的有效性，凸显了其在视觉数据物理属性理解中的关键作用。在线演示、代码、更多案例及标注数据集详见\href{https://Gaussian-Property.github.io}{此链接}。