3D Gaussian Splatting has achieved remarkable success in reconstructing both static and dynamic 3D scenes. However, in a scene represented by 3D Gaussian primitives, interactions between objects suffer from inaccurate 3D segmentation, imprecise deformation among different materials, and severe rendering artifacts. To address these challenges, we introduce PIG: Physically-Based Multi-Material Interaction with 3D Gaussians, a novel approach that combines 3D object segmentation with the simulation of interacting objects in high precision. Firstly, our method facilitates fast and accurate mapping from 2D pixels to 3D Gaussians, enabling precise 3D object-level segmentation. Secondly, we assign unique physical properties to correspondingly segmented objects within the scene for multi-material coupled interactions. Finally, we have successfully embedded constraint scales into deformation gradients, specifically clamping the scaling and rotation properties of the Gaussian primitives to eliminate artifacts and achieve geometric fidelity and visual consistency. Experimental results demonstrate that our method not only outperforms the state-of-the-art (SOTA) in terms of visual quality, but also opens up new directions and pipelines for the field of physically realistic scene generation.

翻译：三维高斯泼溅在静态与动态三维场景重建方面取得了显著成功。然而，在以三维高斯基元表示的场景中，物体间的交互存在三维分割不准确、不同材质间变形不精确以及严重渲染伪影等问题。为应对这些挑战，我们提出PIG：基于物理的多材质三维高斯交互，这是一种将三维物体分割与高精度交互物体模拟相结合的新方法。首先，我们的方法实现了从二维像素到三维高斯的快速准确映射，从而支持精确的三维物体级分割。其次，我们为场景中相应分割的物体赋予独特的物理属性，以实现多材质耦合交互。最后，我们成功将约束尺度嵌入变形梯度中，具体通过钳制高斯基元的缩放与旋转属性来消除伪影，从而实现几何保真度与视觉一致性。实验结果表明，我们的方法不仅在视觉质量上优于现有最佳技术，还为物理真实感场景生成领域开辟了新的方向与技术路径。