In this study, we introduce the DriveEnv-NeRF framework, which leverages Neural Radiance Fields (NeRF) to enable the validation and faithful forecasting of the efficacy of autonomous driving agents in a targeted real-world scene. Standard simulator-based rendering often fails to accurately reflect real-world performance due to the sim-to-real gap, which represents the disparity between virtual simulations and real-world conditions. To mitigate this gap, we propose a workflow for building a high-fidelity simulation environment of the targeted real-world scene using NeRF. This approach is capable of rendering realistic images from novel viewpoints and constructing 3D meshes for emulating collisions. The validation of these capabilities through the comparison of success rates in both simulated and real environments demonstrates the benefits of using DriveEnv-NeRF as a real-world performance indicator. Furthermore, the DriveEnv-NeRF framework can serve as a training environment for autonomous driving agents under various lighting conditions. This approach enhances the robustness of the agents and reduces performance degradation when deployed to the target real scene, compared to agents fully trained using the standard simulator rendering pipeline.

翻译：本研究提出了DriveEnv-NeRF框架，该框架利用神经辐射场（NeRF）技术，实现在特定真实世界场景中对自动驾驶智能体效能进行验证与可靠预测。基于标准模拟器的渲染方法常因仿真与现实间的差距（即虚拟仿真条件与真实世界条件之间的差异）而无法准确反映真实性能。为缓解此差距，我们提出了一种利用NeRF构建特定真实场景高保真仿真环境的工作流程。该方法能够从新视角渲染逼真图像，并构建用于碰撞仿真的三维网格。通过对比模拟环境与真实环境中的成功率验证了这些能力，证明了使用DriveEnv-NeRF作为真实世界性能指标的优势。此外，DriveEnv-NeRF框架可作为自动驾驶智能体在不同光照条件下的训练环境。相较于完全采用标准模拟器渲染流程训练的智能体，该方法增强了智能体的鲁棒性，并降低了其在目标真实场景中部署时的性能衰减。