Autonomous driving simulation system plays a crucial role in enhancing self-driving data and simulating complex and rare traffic scenarios, ensuring navigation safety. However, traditional simulation systems, which often heavily rely on manual modeling and 2D image editing, struggled with scaling to extensive scenes and generating realistic simulation data. In this study, we present S-NeRF++, an innovative autonomous driving simulation system based on neural reconstruction. Trained on widely-used self-driving datasets such as nuScenes and Waymo, S-NeRF++ can generate a large number of realistic street scenes and foreground objects with high rendering quality as well as offering considerable flexibility in manipulation and simulation. Specifically, S-NeRF++ is an enhanced neural radiance field for synthesizing large-scale scenes and moving vehicles, with improved scene parameterization and camera pose learning. The system effectively utilizes noisy and sparse LiDAR data to refine training and address depth outliers, ensuring high-quality reconstruction and novel-view rendering. It also provides a diverse foreground asset bank by reconstructing and generating different foreground vehicles to support comprehensive scenario creation.Moreover, we have developed an advanced foreground-background fusion pipeline that skillfully integrates illumination and shadow effects, further enhancing the realism of our simulations. With the high-quality simulated data provided by our S-NeRF++, we found the perception methods enjoy performance boosts on several autonomous driving downstream tasks, further demonstrating our proposed simulator's effectiveness.

翻译：自动驾驶仿真系统在增强自动驾驶数据、模拟复杂与罕见交通场景以确保导航安全方面发挥着关键作用。然而，传统仿真系统通常严重依赖人工建模与二维图像编辑，难以扩展到大规模场景并生成逼真的仿真数据。本研究提出S-NeRF++，一种基于神经重建的创新性自动驾驶仿真系统。该系统在nuScenes、Waymo等广泛使用的自动驾驶数据集上训练，能够生成大量具有高渲染质量的逼真街景与前景物体，同时在操控与仿真方面提供显著的灵活性。具体而言，S-NeRF++是一种用于合成大规模场景与运动车辆的增强型神经辐射场，其改进了场景参数化与相机姿态学习。该系统有效利用带噪声且稀疏的LiDAR数据优化训练并处理深度异常值，确保高质量重建与新视角渲染。它还通过重建并生成不同的前景车辆构建了多样化的前景资产库，以支持综合性场景创建。此外，我们开发了先进的前景-背景融合管线，巧妙整合光照与阴影效果，进一步提升仿真的真实感。利用S-NeRF++提供的高质量仿真数据，我们发现感知方法在多项自动驾驶下游任务中均获得性能提升，进一步证明了所提仿真器的有效性。