Simulators are indispensable for research in autonomous systems such as self-driving cars, autonomous robots and drones. Despite significant progress in various simulation aspects, such as graphical realism, an evident gap persists between the virtual and real-world environments. Since the ultimate goal is to deploy the autonomous systems in the real world, closing the sim2real gap is of utmost importance. In this paper, we employ a state-of-the-art approach to enhance the photorealism of simulated data, aligning them with the visual characteristics of real-world datasets. Based on this, we developed CARLA2Real, an easy-to-use, publicly available tool (plug-in) for the widely used and open-source CARLA simulator. This tool enhances the output of CARLA in near real-time, achieving a frame rate of 13 FPS, translating it to the visual style and realism of real-world datasets such as Cityscapes, KITTI, and Mapillary Vistas. By employing the proposed tool, we generated synthetic datasets from both the simulator and the enhancement model outputs, including their corresponding ground truth annotations for tasks related to autonomous driving. Then, we performed a number of experiments to evaluate the impact of the proposed approach on feature extraction and semantic segmentation methods when trained on the enhanced synthetic data. The results demonstrate that the sim2real gap is significant and can indeed be reduced by the introduced approach.

翻译：模拟器在自动驾驶汽车、自主机器人和无人机等自主系统的研究中不可或缺。尽管在图形真实感等模拟各方面已取得显著进展，但虚拟环境与现实世界之间仍存在明显差距。由于最终目标是将自主系统部署于现实世界，缩小仿真与现实差距至关重要。本文采用一种先进方法提升模拟数据的照片真实感，使其与现实世界数据集的视觉特征对齐。基于此，我们开发了CARLA2Real——一个易于使用、公开可用的工具（插件），适用于广泛使用的开源CARLA模拟器。该工具以接近实时的13帧/秒速率增强CARLA的输出，将其转换为Cityscapes、KITTI和Mapillary Vistas等现实世界数据集的视觉风格与真实感。通过使用该工具，我们从模拟器和增强模型输出中生成了合成数据集，包含自动驾驶相关任务对应的真实标注。随后，我们进行了一系列实验，评估所提方法对基于增强合成数据训练的特征提取与语义分割方法的影响。结果表明，仿真与现实差距确实显著，且通过引入的方法可以有效缩小这一差距。