Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between synthetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data. Videos can be found on our project page: https://splatsim.github.io

翻译：仿真到现实迁移，特别是依赖RGB图像的操作策略，由于合成视觉数据与真实世界视觉数据之间存在显著领域偏移，仍然是机器人学中的一个关键挑战。本文提出SplatSim，一种新颖的框架，它利用高斯溅射作为主要渲染基元，以减小基于RGB的操作策略的仿真到现实差距。通过在仿真器中用高斯溅射替代传统的网格表示，SplatSim能够生成高度逼真的合成数据，同时保持仿真的可扩展性和成本效益。我们通过在SplatSim中训练操作策略并以零样本方式将其部署到现实世界，证明了我们框架的有效性，平均成功率达到了86.25%，而使用真实世界数据训练的策略成功率为97.5%。相关视频可在我们的项目页面查看：https://splatsim.github.io