Labeling LiDAR point clouds for training autonomous driving is extremely expensive and difficult. LiDAR simulation aims at generating realistic LiDAR data with labels for training and verifying self-driving algorithms more efficiently. Recently, Neural Radiance Fields (NeRF) have been proposed for novel view synthesis using implicit reconstruction of 3D scenes. Inspired by this, we present NeRF-LIDAR, a novel LiDAR simulation method that leverages real-world information to generate realistic LIDAR point clouds. Different from existing LiDAR simulators, we use real images and point cloud data collected by self-driving cars to learn the 3D scene representation, point cloud generation and label rendering. We verify the effectiveness of our NeRF-LiDAR by training different 3D segmentation models on the generated LiDAR point clouds. It reveals that the trained models are able to achieve similar accuracy when compared with the same model trained on the real LiDAR data. Besides, the generated data is capable of boosting the accuracy through pre-training which helps reduce the requirements of the real labeled data.

翻译：摘要：为训练自动驾驶系统标注激光雷达点云数据极为昂贵且困难。激光雷达模拟旨在生成带有标签的逼真激光雷达数据，以更高效地训练和验证自动驾驶算法。近年来，神经辐射场（NeRF）通过三维场景的隐式重建被提出用于新视角合成。受此启发，我们提出了NeRF-LiDAR——一种利用真实世界信息生成逼真激光雷达点云的新型激光雷达模拟方法。与现有激光雷达模拟器不同，我们使用自动驾驶车辆采集的真实图像和点云数据来学习三维场景表示、点云生成及标签渲染。通过在生成的激光雷达点云上训练不同的三维分割模型，我们验证了NeRF-LiDAR的有效性。结果表明，与在真实激光雷达数据上训练的同一模型相比，所训练模型能够达到相近的精度。此外，通过预训练，生成数据还能提升模型精度，从而有助于减少对真实标注数据的需求。