Scanning real-life scenes with modern registration devices typically gives incomplete point cloud representations, primarily due to the limitations of partial scanning, 3D occlusions, and dynamic light conditions. Recent works on processing incomplete point clouds have always focused on point cloud completion. However, these approaches do not ensure consistency between the completed point cloud and the captured images regarding color and geometry. We propose using Generative Point-based NeRF (GPN) to reconstruct and repair a partial cloud by fully utilizing the scanning images and the corresponding reconstructed cloud. The repaired point cloud can achieve multi-view consistency with the captured images at high spatial resolution. For the finetunes of a single scene, we optimize the global latent condition by incorporating an Auto-Decoder architecture while retaining multi-view consistency. As a result, the generated point clouds are smooth, plausible, and geometrically consistent with the partial scanning images. Extensive experiments on ShapeNet demonstrate that our works achieve competitive performances to the other state-of-the-art point cloud-based neural scene rendering and editing performances.

翻译：使用现代配准设备扫描真实场景时，通常因部分扫描、三维遮挡及动态光照条件的限制，会得到不完整的点云表示。近期关于处理不完整点云的研究主要聚焦于点云补全，但这些方法无法确保补全后的点云与捕获图像在颜色和几何形态上的一致性。我们提出采用基于生成式点的神经辐射场（Generative Point-based NeRF，GPN），通过充分利用扫描图像及其对应的重建点云，对部分点云进行重建与修复。修复后的点云能够以高空间分辨率实现与捕获图像的多视图一致性。针对单场景的微调，我们在保留多视图一致性的同时，通过引入自动解码器架构优化全局隐式条件。由此生成的点云不仅平滑合理，且与部分扫描图像在几何上保持一致性。在ShapeNet数据集上的大量实验表明，我们的方法在基于点云的神经场景渲染与编辑任务中，性能表现出与当前最先进方法相当的竞争力。