In this study, we introduce the DriveEnv-NeRF framework, which leverages Neural Radiance Fields (NeRF) to enable the validation and faithful forecasting of the efficacy of autonomous driving agents in a targeted real-world scene. Standard simulator-based rendering often fails to accurately reflect real-world performance due to the sim-to-real gap, which represents the disparity between virtual simulations and real-world conditions. To mitigate this gap, we propose a workflow for building a high-fidelity simulation environment of the targeted real-world scene using NeRF. This approach is capable of rendering realistic images from novel viewpoints and constructing 3D meshes for emulating collisions. The validation of these capabilities through the comparison of success rates in both simulated and real environments demonstrates the benefits of using DriveEnv-NeRF as a real-world performance indicator. Furthermore, the DriveEnv-NeRF framework can serve as a training environment for autonomous driving agents under various lighting conditions. This approach enhances the robustness of the agents and reduces performance degradation when deployed to the target real scene, compared to agents fully trained using the standard simulator rendering pipeline.

翻译：本研究提出DriveEnv-NeRF框架，通过利用神经辐射场（NeRF）实现对自动驾驶智能体在特定真实场景中效能的有效验证与可靠预测。标准基于模拟器的渲染技术常因仿真-现实差距（即虚拟仿真与真实环境之间的差异）而无法准确反映真实世界性能。为缩小这一差距，我们提出一种工作流程，采用NeRF构建目标真实场景的高保真仿真环境。该方法能够从新视角渲染逼真图像，并构建用于碰撞模拟的三维网格。通过对比模拟环境与真实环境中的成功率，验证了该能力，证明了DriveEnv-NeRF作为真实世界性能指标的有效性。此外，DriveEnv-NeRF框架可作为自动驾驶智能体在不同光照条件下的训练环境。相较于完全使用标准模拟器渲染管线训练的智能体，本方法增强了智能体的鲁棒性，并降低了部署至真实场景时的性能衰减。