Labeling LiDAR point clouds for training autonomous driving is extremely expensive and difficult. LiDAR simulation aims at generating realistic LiDAR data with labels for training and verifying self-driving algorithms more efficiently. Recently, Neural Radiance Fields (NeRF) have been proposed for novel view synthesis using implicit reconstruction of 3D scenes. Inspired by this, we present NeRF-LIDAR, a novel LiDAR simulation method that leverages real-world information to generate realistic LIDAR point clouds. Different from existing LiDAR simulators, we use real images and point cloud data collected by self-driving cars to learn the 3D scene representation, point cloud generation and label rendering. We verify the effectiveness of our NeRF-LiDAR by training different 3D segmentation models on the generated LiDAR point clouds. It reveals that the trained models are able to achieve similar accuracy when compared with the same model trained on the real LiDAR data. Besides, the generated data is capable of boosting the accuracy through pre-training which helps reduce the requirements of the real labeled data.

翻译：为自动驾驶训练标注LiDAR点云数据成本极高且难度巨大。LiDAR仿真旨在生成带有标注的真实感LiDAR数据，以更高效地训练和验证自动驾驶算法。近期，神经辐射场（NeRF）通过三维场景隐式重建实现新视角合成。受此启发，我们提出NeRF-LiDAR——一种利用真实世界信息生成真实感LiDAR点云的新型LiDAR仿真方法。与现有LiDAR仿真器不同，该方法采用自动驾驶车辆采集的真实图像与点云数据，学习三维场景表征、点云生成与标签渲染。通过在生成的LiDAR点云上训练不同的3D分割模型，我们验证了NeRF-LiDAR的有效性。结果表明，与使用真实LiDAR数据训练的同类模型相比，该框架训练的模型可获得相近精度。此外，生成的LiDAR数据可通过预训练机制提升模型精度，从而降低对真实标注数据的需求。