As consumer Virtual Reality (VR) and Mixed Reality (MR) technologies gain momentum, there's a growing focus on the development of engagements with 3D virtual content. Unfortunately, traditional techniques for content creation, editing, and interaction within these virtual spaces are fraught with difficulties. They tend to be not only engineering-intensive but also require extensive expertise, which adds to the frustration and inefficiency in virtual object manipulation. Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction, offering a seamless and intuitive user experience. By developing a physical dynamics-aware interactive Gaussian Splatting in a Virtual Reality setting, and constructing a highly efficient two-level embedding strategy alongside deformable body simulations, VR-GS ensures real-time execution with highly realistic dynamic responses. The components of our Virtual Reality system are designed for high efficiency and effectiveness, starting from detailed scene reconstruction and object segmentation, advancing through multi-view image in-painting, and extending to interactive physics-based editing. The system also incorporates real-time deformation embedding and dynamic shadow casting, ensuring a comprehensive and engaging virtual experience.Our project page is available at: https://yingjiang96.github.io/VR-GS/.

翻译：随着消费级虚拟现实（VR）与混合现实（MR）技术的蓬勃发展，三维虚拟内容的交互体验成为研究热点。然而，现有虚拟空间中内容创建、编辑与交互的传统技术存在诸多困难：不仅工程复杂度高，而且需要大量专业知识，导致虚拟物体操作效率低下且体验不佳。我们提出的VR-GS系统在人机交互式三维内容领域实现了重大突破，提供无缝且直观的沉浸式体验。通过在虚拟现实环境中开发具有物理动力学感知的交互式高斯泼溅（Gaussian Splatting）技术，并构建高效的双层嵌入策略与可变形体仿真框架，VR-GS系统实现了实时运算与高保真动态响应。系统各组件均兼顾效率与性能：从场景精准重建与物体分割开始，经多视图图像修复技术处理，再到基于物理的交互式编辑功能，最终集成实时变形嵌入与动态阴影投射模块，打造全方位、高沉浸的虚拟交互体验。项目主页：https://yingjiang96.github.io/VR-GS/。