Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between synthetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data. Videos can be found on our project page: https://splatsim.github.io

翻译：仿真到现实迁移，特别是对于依赖RGB图像的操控策略，由于合成视觉数据与真实世界视觉数据之间存在显著领域偏移，仍然是机器人学中的一个关键挑战。本文提出SplatSim，一种新颖的框架，利用高斯溅射作为主要渲染基元，以减少基于RGB的操控策略的仿真到现实差距。通过在仿真器中用高斯溅射替代传统的网格表示，SplatSim能够生成高度逼真的合成数据，同时保持仿真的可扩展性和成本效益。我们通过在SplatSim中训练操控策略并以零样本方式部署到现实世界来证明我们框架的有效性，实现了平均86.25%的成功率，相比之下，在真实世界数据上训练的策略成功率为97.5%。视频可在我们的项目页面找到：https://splatsim.github.io