Learning accurate and parsimonious point cloud representations of scene surfaces from scratch remains a challenge in 3D representation learning. Existing point-based methods often suffer from the vanishing gradient problem or require a large number of points to accurately model scene geometry and texture. To address these limitations, we propose Proximity Attention Point Rendering (PAPR), a novel method that consists of a point-based scene representation and a differentiable renderer. Our scene representation uses a point cloud where each point is characterized by its spatial position, influence score, and view-independent feature vector. The renderer selects the relevant points for each ray and produces accurate colours using their associated features. PAPR effectively learns point cloud positions to represent the correct scene geometry, even when the initialization drastically differs from the target geometry. Notably, our method captures fine texture details while using only a parsimonious set of points. We also demonstrate four practical applications of our method: zero-shot geometry editing, object manipulation, texture transfer, and exposure control. More results and code are available on our project website at https://zvict.github.io/papr/.

翻译：摘要：从零开始学习准确且精简的场景表面点云表示，仍是3D表示学习中的一项挑战。现有基于点的方法常面临梯度消失问题，或需要大量点才能精确建模场景几何与纹理。为克服这些局限，我们提出邻近注意力点渲染（PAPR），一种包含基于点的场景表示与可微分渲染器的新方法。我们场景表示采用点云，每个点由空间位置、影响分数和视角无关特征向量表征。渲染器为每条光线选择相关点，并利用其关联特征生成准确颜色。即使初始位置与目标几何存在显著差异，PAPR也能有效学习点云位置以表征正确场景几何。值得注意的是，本方法仅使用精简点集即可捕捉精细纹理细节。我们还展示了该方法在零样本几何编辑、物体操控、纹理迁移和曝光控制四项实用场景中的应用。更多结果与代码详见项目网站 https://zvict.github.io/papr/。