Thoroughly testing autonomy systems is crucial in the pursuit of safe autonomous driving vehicles. It necessitates creating safety-critical scenarios that go beyond what can be safely collected from real-world data, as many of these scenarios occur infrequently on public roads. However, the evaluation of most existing NVS methods relies on sporadic sampling of image frames from the training data, comparing the rendered images with ground truth images using metrics. Unfortunately, this evaluation protocol falls short of meeting the actual requirements in closed-loop simulations. Specifically, the true application demands the capability to render novel views that extend beyond the original trajectory (such as cross-lane views), which are challenging to capture in the real world. To address this, this paper presents a novel driving view synthesis dataset and benchmark specifically designed for autonomous driving simulations. This dataset is unique as it includes testing images captured by deviating from the training trajectory by 1-4 meters. It comprises six sequences encompassing various time and weather conditions. Each sequence contains 450 training images, 150 testing images, and their corresponding camera poses and intrinsic parameters. Leveraging this novel dataset, we establish the first realistic benchmark for evaluating existing NVS approaches under front-only and multi-camera settings. The experimental findings underscore the significant gap that exists in current approaches, revealing their inadequate ability to fulfill the demanding prerequisites of cross-lane or closed-loop simulation. Our dataset is released publicly at the project page: https://3d-aigc.github.io/XLD/.

翻译：全面测试自动驾驶系统对于实现安全自动驾驶车辆至关重要。这需要创建超出从现实世界数据中安全收集范围的安全关键场景，因为许多此类场景在公共道路上极少发生。然而，大多数现有神经视角合成（NVS）方法的评估依赖于从训练数据中随机采样图像帧，并使用度量指标比较渲染图像与真实图像。遗憾的是，这种评估方案无法满足闭环模拟中的实际需求。具体而言，真实应用需要能够渲染超出原始轨迹的新视角（例如跨车道视角），而这些视角在现实世界中难以捕获。为此，本文提出了一个专为自动驾驶模拟设计的新型驾驶视角合成数据集和基准。该数据集的独特之处在于包含了偏离训练轨迹1-4米拍摄的测试图像。它包含六个序列，涵盖不同时间和天气条件。每个序列包含450张训练图像、150张测试图像及其对应的相机姿态和内在参数。利用这一新颖数据集，我们建立了首个在单前视和多相机设置下评估现有NVS方法的现实基准。实验结果突显了当前方法存在的显著差距，揭示了它们无法满足跨车道或闭环模拟的严苛要求。我们的数据集已在项目页面公开发布：https://3d-aigc.github.io/XLD/。