As consumer Virtual Reality (VR) and Mixed Reality (MR) technologies gain momentum, there's a growing focus on the development of engagements with 3D virtual content. Unfortunately, traditional techniques for content creation, editing, and interaction within these virtual spaces are fraught with difficulties. They tend to be not only engineering-intensive but also require extensive expertise, which adds to the frustration and inefficiency in virtual object manipulation. Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction, offering a seamless and intuitive user experience. By developing a physical dynamics-aware interactive Gaussian Splatting in a Virtual Reality setting, and constructing a highly efficient two-level embedding strategy alongside deformable body simulations, VR-GS ensures real-time execution with highly realistic dynamic responses. The components of our Virtual Reality system are designed for high efficiency and effectiveness, starting from detailed scene reconstruction and object segmentation, advancing through multi-view image in-painting, and extending to interactive physics-based editing. The system also incorporates real-time deformation embedding and dynamic shadow casting, ensuring a comprehensive and engaging virtual experience.Our project page is available at: https://yingjiang96.github.io/VR-GS/.

翻译：随着消费级虚拟现实（VR）与混合现实（MR）技术的蓬勃发展，三维虚拟内容交互的开发日益受到关注。然而，传统虚拟空间中的内容创作、编辑与交互技术仍面临诸多挑战：不仅工程复杂度高，还需大量专业知识，导致虚拟物体操作效率低下且充满挫败感。我们提出的VR-GS系统代表了以人为中心的三维内容交互领域的重大突破，能够提供无缝且直观的用户体验。通过在虚拟现实环境中开发物理动力学感知的交互式高斯泼溅技术，并构建高效的双层嵌入策略与可变形物体仿真框架，VR-GS系统确保了实时运行与高度逼真的动态响应。该虚拟现实系统的各组件均以高效性和有效性为设计目标：从场景精细重建与物体分割，到多视角图像修复，再到基于物理的交互式编辑，均实现了技术进阶。系统还集成了实时变形嵌入与动态阴影投射功能，从而提供全面且沉浸式的虚拟体验。我们的项目页面访问地址为：https://yingjiang96.github.io/VR-GS/。