We introduce Physically Enhanced Gaussian Splatting Simulation System (PEGASUS) for 6DOF object pose dataset generation, a versatile dataset generator based on 3D Gaussian Splatting. Environment and object representations can be easily obtained using commodity cameras to reconstruct with Gaussian Splatting. PEGASUS allows the composition of new scenes by merging the respective underlying Gaussian Splatting point cloud of an environment with one or multiple objects. Leveraging a physics engine enables the simulation of natural object placement within a scene through interaction between meshes extracted for the objects and the environment. Consequently, an extensive amount of new scenes - static or dynamic - can be created by combining different environments and objects. By rendering scenes from various perspectives, diverse data points such as RGB images, depth maps, semantic masks, and 6DoF object poses can be extracted. Our study demonstrates that training on data generated by PEGASUS enables pose estimation networks to successfully transfer from synthetic data to real-world data. Moreover, we introduce the Ramen dataset, comprising 30 Japanese cup noodle items. This dataset includes spherical scans that captures images from both object hemisphere and the Gaussian Splatting reconstruction, making them compatible with PEGASUS.

翻译：我们提出物理增强高斯泼溅仿真系统（PEGASUS），这是一种基于三维高斯泼溅的通用数据集生成器，专为6自由度物体姿态数据集生成而设计。利用消费级相机即可轻松获取场景与物体的表征，并通过高斯泼溅技术进行重建。PEGASUS通过合并场景与单个或多个物体的底层高斯泼溅点云，实现新场景的合成。借助物理引擎，通过物体与场景提取的网格之间的交互，可以模拟物体在场景中的自然放置。因此，通过组合不同的环境与物体，能够生成海量静态或动态的新场景。通过从不同视角渲染场景，可提取RGB图像、深度图、语义掩码及6自由度物体姿态等多样化数据。研究表明，基于PEGASUS生成的数据进行训练，可使姿态估计网络成功实现从合成数据到真实数据的迁移。此外，我们引入包含30种日本杯面物品的Ramen数据集。该数据集包含球面扫描数据，可捕捉物体半球图像及高斯泼溅重建结果，使其与PEGASUS兼容。