Synthesizing photo-realistic images from a point cloud is challenging because of the sparsity of point cloud representation. Recent Neural Radiance Fields and extensions are proposed to synthesize realistic images from 2D input. In this paper, we present Point2Pix as a novel point renderer to link the 3D sparse point clouds with 2D dense image pixels. Taking advantage of the point cloud 3D prior and NeRF rendering pipeline, our method can synthesize high-quality images from colored point clouds, generally for novel indoor scenes. To improve the efficiency of ray sampling, we propose point-guided sampling, which focuses on valid samples. Also, we present Point Encoding to build Multi-scale Radiance Fields that provide discriminative 3D point features. Finally, we propose Fusion Encoding to efficiently synthesize high-quality images. Extensive experiments on the ScanNet and ArkitScenes datasets demonstrate the effectiveness and generalization.

翻译：从点云合成照片级真实感图像极具挑战性，因为点云表示的稀疏性。近期提出的神经辐射场及其扩展方法可从二维输入合成真实感图像。本文提出新型点渲染器Point2Pix，用于连接三维稀疏点云与二维密集图像像素。该方法利用点云三维先验及NeRF渲染管线，可从彩色点云中为新颖室内场景合成高质量图像。为提升光线采样效率，我们提出点引导采样策略，聚焦有效样本。同时引入点编码技术构建多尺度辐射场，提供具有判别性的三维点特征。最后提出融合编码机制以高效合成高质量图像。在ScanNet和ArkitScenes数据集上的大量实验证明了该方法的有效性与泛化能力。