LiDAR provides accurate geometric measurements of the 3D world. Unfortunately, dense LiDARs are very expensive and the point clouds captured by low-beam LiDAR are often sparse. To address these issues, we present UltraLiDAR, a data-driven framework for scene-level LiDAR completion, LiDAR generation, and LiDAR manipulation. The crux of UltraLiDAR is a compact, discrete representation that encodes the point cloud's geometric structure, is robust to noise, and is easy to manipulate. We show that by aligning the representation of a sparse point cloud to that of a dense point cloud, we can densify the sparse point clouds as if they were captured by a real high-density LiDAR, drastically reducing the cost. Furthermore, by learning a prior over the discrete codebook, we can generate diverse, realistic LiDAR point clouds for self-driving. We evaluate the effectiveness of UltraLiDAR on sparse-to-dense LiDAR completion and LiDAR generation. Experiments show that densifying real-world point clouds with our approach can significantly improve the performance of downstream perception systems. Compared to prior art on LiDAR generation, our approach generates much more realistic point clouds. According to A/B test, over 98.5\% of the time human participants prefer our results over those of previous methods.

翻译：LiDAR提供了3D世界的精确几何测量。然而，高密度LiDAR成本高昂，而低线束LiDAR捕获的点云通常较为稀疏。为解决这些问题，我们提出UltraLiDAR——一个面向场景级LiDAR补全、生成与操控的数据驱动框架。UltraLiDAR的核心是一种紧致的离散表示，该表示编码了点云的几何结构，对噪声具有鲁棒性且易于操控。研究表明，通过将稀疏点云的表示与稠密点云的表示对齐，我们能够将稀疏点云稠密化，使其如同由真实高密度LiDAR捕获一般，从而大幅降低成本。此外，通过学习离散码本上的先验分布，我们可以为自动驾驶生成多样化且逼真的LiDAR点云。我们在稀疏到稠密LiDAR补全与LiDAR生成任务上评估了UltraLiDAR的有效性。实验表明，使用我们的方法对真实世界点云进行稠密化能显著提升下游感知系统的性能。相较于LiDAR生成的现有技术，我们的方法能生成更为逼真的点云。A/B测试显示，超过98.5%的情况下，人类参与者更偏好我们的结果而非先前方法。