We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering. Motivated by the fact that informative point cloud features should be able to encode rich geometry and appearance cues and render realistic images, we train a point-cloud encoder within a devised point-based neural renderer by comparing the rendered images with real images on massive RGB-D data. The learned point-cloud encoder can be easily integrated into various downstream tasks, including not only high-level tasks like 3D detection and segmentation, but low-level tasks like 3D reconstruction and image synthesis. Extensive experiments on various tasks demonstrate the superiority of our approach compared to existing pre-training methods.

翻译：摘要：我们提出了一种通过可微分神经渲染进行点云表征自监督学习的新方法。基于如下动机：包含信息的点云特征应能够编码丰富的几何与外观线索，并渲染出逼真的图像，我们通过在大规模RGB-D数据上将渲染图像与真实图像进行对比，在设计的基于点的神经渲染器中训练点云编码器。学习得到的点云编码器可轻松集成至多种下游任务，不仅包括3D检测与分割等高层任务，还涵盖3D重建与图像合成等低层任务。在各类任务上的大量实验表明，相较于现有预训练方法，我们的方法具有优越性。