Thoroughly testing autonomy systems is crucial in the pursuit of safe autonomous driving vehicles. It necessitates creating safety-critical scenarios that go beyond what can be safely collected from real-world data, as many of these scenarios occur infrequently on public roads. However, the evaluation of most existing NVS methods relies on sporadic sampling of image frames from the training data, comparing the rendered images with ground truth images using metrics. Unfortunately, this evaluation protocol falls short of meeting the actual requirements in closed-loop simulations. Specifically, the true application demands the capability to render novel views that extend beyond the original trajectory (such as cross-lane views), which are challenging to capture in the real world. To address this, this paper presents a novel driving view synthesis dataset and benchmark specifically designed for autonomous driving simulations. This dataset is unique as it includes testing images captured by deviating from the training trajectory by 1-4 meters. It comprises six sequences encompassing various time and weather conditions. Each sequence contains 450 training images, 150 testing images, and their corresponding camera poses and intrinsic parameters. Leveraging this novel dataset, we establish the first realistic benchmark for evaluating existing NVS approaches under front-only and multi-camera settings. The experimental findings underscore the significant gap that exists in current approaches, revealing their inadequate ability to fulfill the demanding prerequisites of cross-lane or closed-loop simulation. Our dataset is released publicly at the project page: https://3d-aigc.github.io/XLD/.

翻译：全面测试自动驾驶系统对于实现安全自动驾驶车辆至关重要。这需要创建超出从真实世界数据中安全收集范围的安全关键场景，因为许多此类场景在公共道路上极少发生。然而，现有大多数神经视角合成（NVS）方法的评估依赖于从训练数据中随机采样图像帧，并使用指标将渲染图像与真实图像进行比较。遗憾的是，这种评估方案无法满足闭环仿真的实际需求。具体而言，实际应用要求能够渲染超出原始轨迹的新视角（例如跨车道视角），而这些视角在现实世界中难以捕捉。为此，本文提出了一个专为自动驾驶仿真设计的新型驾驶视角合成数据集与基准。该数据集的独特之处在于包含了偏离训练轨迹1-4米所捕获的测试图像。它包含六个序列，涵盖不同时间和天气条件。每个序列包含450张训练图像、150张测试图像及其对应的相机姿态与内参。利用这一新颖数据集，我们建立了首个在单前视与多相机设置下评估现有NVS方法的真实基准。实验结果凸显了现有方法存在的显著差距，揭示了其难以满足跨车道或闭环仿真的严苛要求。我们的数据集已在项目页面公开发布：https://3d-aigc.github.io/XLD/。