Predicting how the world can evolve in the future is crucial for motion planning in autonomous systems. Classical methods are limited because they rely on costly human annotations in the form of semantic class labels, bounding boxes, and tracks or HD maps of cities to plan their motion and thus are difficult to scale to large unlabeled datasets. One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences. We show that this task requires algorithms to implicitly capture (1) sensor extrinsics (i.e., the egomotion of the autonomous vehicle), (2) sensor intrinsics (i.e., the sampling pattern specific to the particular LiDAR sensor), and (3) the shape and motion of other objects in the scene. But autonomous systems should make predictions about the world and not their sensors. To this end, we factor out (1) and (2) by recasting the task as one of spacetime (4D) occupancy forecasting. But because it is expensive to obtain ground-truth 4D occupancy, we render point cloud data from 4D occupancy predictions given sensor extrinsics and intrinsics, allowing one to train and test occupancy algorithms with unannotated LiDAR sequences. This also allows one to evaluate and compare point cloud forecasting algorithms across diverse datasets, sensors, and vehicles.

翻译：预测世界未来如何演化对于自主系统中的运动规划至关重要。传统方法受限于需依赖昂贵的人工标注（如语义类别标签、边界框、轨迹或城市高清地图）来规划运动，因而难以扩展至大规模未标注数据集。一种有前景的自监督任务是从未标注的激光雷达序列进行3D点云预测。我们表明，该任务要求算法隐式捕捉：（1）传感器外参（即自主车辆的自我运动）、（2）传感器内参（即特定激光雷达的采样模式），以及（3）场景中其他物体的形状与运动。但自主系统应预测世界状态而非其传感器状态。为此，我们通过将任务重新定义为时空（4D）占用预测，从而分解出（1）和（2）因素。鉴于获取真实4D占用的成本高昂，我们根据传感器外参与内参从4D占用预测中渲染点云数据，从而支持利用未标注激光雷达序列训练和测试占用算法。这一方法还使得跨不同数据集、传感器和车辆评估与比较点云预测算法成为可能。