This paper presents GSWorld, a robust, photo-realistic simulator for robotics manipulation that combines 3D Gaussian Splatting with physics engines. Our framework advocates "closing the loop" of developing manipulation policies with reproducible evaluation of policies learned from real-robot data and sim2real policy training without using real robots. To enable photo-realistic rendering of diverse scenes, we propose a new asset format, which we term GSDF (Gaussian Scene Description File), that infuses Gaussian-on-Mesh representation with robot URDF and other objects. With a streamlined reconstruction pipeline, we curate a database of GSDF that contains 3 robot embodiments for single-arm and bimanual manipulation, as well as more than 40 objects. Combining GSDF with physics engines, we demonstrate several immediate interesting applications: (1) learning zero-shot sim2real pixel-to-action manipulation policy with photo-realistic rendering, (2) automated high-quality DAgger data collection for adapting policies to deployment environments, (3) reproducible benchmarking of real-robot manipulation policies in simulation, (4) simulation data collection by virtual teleoperation, and (5) zero-shot sim2real visual reinforcement learning. Website: https://3dgsworld.github.io/.

翻译：本文提出GSWorld，一个结合3D高斯泼溅与物理引擎的鲁棒、逼真机器人操作模拟器。我们的框架倡导通过真实机器人数据学习策略的可复现评估以及无需真实机器人的模拟到真实策略训练，实现操作策略开发的“闭环”。为实现多样化场景的逼真渲染，我们提出一种新资产格式——GSDF（高斯场景描述文件），该格式将网格上的高斯表示与机器人URDF及其他对象相融合。通过简化的重建流程，我们构建了包含3种单臂与双臂操作机器人实体以及超过40个对象的GSDF数据库。结合GSDF与物理引擎，我们展示了若干即时应用：（1）通过逼真渲染学习零样本模拟到真实的像素到动作操作策略；（2）自动化高质量DAgger数据收集以适配部署环境策略；（3）在模拟中对真实机器人操作策略进行可复现基准测试；（4）通过虚拟遥操作收集模拟数据；（5）零样本模拟到真实视觉强化学习。项目网站：https://3dgsworld.github.io/。