To achieve strong real world performance, neural networks must be trained on large, diverse datasets; however, obtaining and annotating such datasets is costly and time-consuming, particularly for 3D point clouds. In this paper, we describe Paved2Paradise, a simple, cost-effective approach for generating fully labeled, diverse, and realistic lidar datasets from scratch, all while requiring minimal human annotation. Our key insight is that, by deliberately collecting separate "background" and "object" datasets (i.e., "factoring the real world"), we can intelligently combine them to produce a combinatorially large and diverse training set. The Paved2Paradise pipeline thus consists of four steps: (1) collecting copious background data, (2) recording individuals from the desired object class(es) performing different behaviors in an isolated environment (like a parking lot), (3) bootstrapping labels for the object dataset, and (4) generating samples by placing objects at arbitrary locations in backgrounds. To demonstrate the utility of Paved2Paradise, we generated synthetic datasets for two tasks: (1) human detection in orchards (a task for which no public data exists) and (2) pedestrian detection in urban environments. Qualitatively, we find that a model trained exclusively on Paved2Paradise synthetic data is highly effective at detecting humans in orchards, including when individuals are heavily occluded by tree branches. Quantitatively, a model trained on Paved2Paradise data that sources backgrounds from KITTI performs comparably to a model trained on the actual dataset. These results suggest the Paved2Paradise synthetic data pipeline can help accelerate point cloud model development in sectors where acquiring lidar datasets has previously been cost-prohibitive.

翻译：为了在现实世界中取得优异性能，神经网络必须在规模庞大且多样化的数据集上进行训练。然而，获取并标注此类数据集（尤其是三维点云数据）成本高昂且耗时费力。本文提出了一种名为Paved2Paradise的简洁、经济高效的方法，能够在仅需极少人工标注的条件下，从零生成具有完整标注、多样且逼真的激光雷达数据集。我们的核心洞见在于：通过有意收集分离的"背景"与"物体"数据集（即"分解现实世界"），可以智能组合两者，从而生成组合级数量的多样化训练集。Paved2Paradise流程包含四个步骤：（1）收集大量背景数据；（2）在隔离环境（如停车场）中记录目标类别个体执行不同行为的数据；（3）为物体数据集自举标注；（4）通过将物体随机放置于背景中生成样本。为验证Paved2Paradise的实用性，我们针对两个任务生成了合成数据集：（1）果园中的人类检测（该任务尚无公开数据）以及（2）城市环境中的行人检测。定性实验表明，仅使用Paved2Paradise合成数据训练的模型在果园中检测人类时表现优异，尤其当个体被树枝严重遮挡时仍保持高检测效能。定量实验显示，若从KITTI数据集中获取背景并基于Paved2Paradise数据训练模型，其性能与直接在原始数据集上训练的模型相当。这些结果表明，Paved2Paradise合成数据管道能够加速激光雷达数据集获取成本居高不下的领域中点云模型的发展进程。