We aim to generate fine-grained 3D geometry from large-scale sparse LiDAR scans, abundantly captured by autonomous vehicles (AV). Contrary to prior work on AV scene completion, we aim to extrapolate fine geometry from unlabeled and beyond spatial limits of LiDAR scans, taking a step towards generating realistic, high-resolution simulation-ready 3D street environments. We propose hierarchical Generative Cellular Automata (hGCA), a spatially scalable conditional 3D generative model, which grows geometry recursively with local kernels following, in a coarse-to-fine manner, equipped with a light-weight planner to induce global consistency. Experiments on synthetic scenes show that hGCA generates plausible scene geometry with higher fidelity and completeness compared to state-of-the-art baselines. Our model generalizes strongly from sim-to-real, qualitatively outperforming baselines on the Waymo-open dataset. We also show anecdotal evidence of the ability to create novel objects from real-world geometric cues even when trained on limited synthetic content. More results and details can be found on https://research.nvidia.com/labs/toronto-ai/hGCA/.

翻译：本文旨在从自动驾驶车辆大量采集的大规模稀疏激光雷达扫描中生成精细的三维几何结构。与先前自动驾驶场景补全的研究不同，我们的目标是从无标注且超出激光雷达扫描空间范围的区域外推出精细几何，从而向生成逼真、高分辨率、可用于仿真的三维街道环境迈进一步。我们提出了分层生成式细胞自动机（hGCA），一种空间可扩展的条件式三维生成模型。该模型通过局部核函数以从粗到细的方式递归生成几何结构，并配备轻量级规划器以保证全局一致性。在合成场景上的实验表明，与现有先进基线方法相比，hGCA能够以更高的保真度和完整性生成合理的场景几何。我们的模型在从仿真到真实场景的迁移中表现出强大的泛化能力，在Waymo开放数据集上定性评估优于基线方法。我们还提供了实例证据，表明模型即使仅在有限合成数据上训练，也能根据真实世界的几何线索生成新颖物体。更多结果与细节请访问：https://research.nvidia.com/labs/toronto-ai/hGCA/。