To achieve strong real world performance, neural networks must be trained on large, diverse datasets; however, obtaining and annotating such datasets is costly and time-consuming, particularly for 3D point clouds. In this paper, we describe Paved2Paradise, a simple, cost-effective approach for generating fully labeled, diverse, and realistic lidar datasets from scratch, all while requiring minimal human annotation. Our key insight is that, by deliberately collecting separate "background" and "object" datasets (i.e., "factoring the real world"), we can intelligently combine them to produce a combinatorially large and diverse training set. The Paved2Paradise pipeline thus consists of four steps: (1) collecting copious background data, (2) recording individuals from the desired object class(es) performing different behaviors in an isolated environment (like a parking lot), (3) bootstrapping labels for the object dataset, and (4) generating samples by placing objects at arbitrary locations in backgrounds. To demonstrate the utility of Paved2Paradise, we generated synthetic datasets for two tasks: (1) human detection in orchards (a task for which no public data exists) and (2) pedestrian detection in urban environments. Qualitatively, we find that a model trained exclusively on Paved2Paradise synthetic data is highly effective at detecting humans in orchards, including when individuals are heavily occluded by tree branches. Quantitatively, a model trained on Paved2Paradise data that sources backgrounds from KITTI performs comparably to a model trained on the actual dataset. These results suggest the Paved2Paradise synthetic data pipeline can help accelerate point cloud model development in sectors where acquiring lidar datasets has previously been cost-prohibitive.

翻译：为使神经网络在真实世界中表现优异，必须使用大规模、多样化的数据集进行训练；然而，获取并标注此类数据集（尤其是三维点云）成本高昂且耗时。本文提出Paved2Paradise——一种简单、经济高效的方法，可从零生成完全标注、多样化且逼真的激光雷达数据集，且仅需极少的人工标注。我们的核心见解是：通过有意识地分离采集"背景"和"物体"数据集（即"分解真实世界"），可以智能组合这些数据，从而生成组合规模庞大且多样化的训练集。Paved2Paradise流程包含四个步骤：(1) 采集大量背景数据，(2) 在隔离环境（如停车场）中记录目标物体类别个体执行不同行为的数据，(3) 为物体数据集启动标注，以及(4) 通过在背景中任意位置放置物体来生成样本。为展示Paved2Paradise的实用性，我们为两个任务生成了合成数据集：(1) 果园中的人体检测（该任务尚无公开数据集），(2) 城市环境中的行人检测。定性结果表明，仅使用Paved2Paradise合成数据训练的模型在果园人体检测中表现优异，即使在个体被树枝严重遮挡时依然有效。定量结果显示，使用Paved2Paradise数据（背景源自KITTI）训练的模型与在真实数据集上训练的模型性能相当。这些结果表明，Paved2Paradise合成数据流程可助力加速那些因获取激光雷达数据集成本过高而受限领域的点云模型开发。