This paper introduces a novel paradigm for the generalizable neural radiance field (NeRF). Previous generic NeRF methods combine multiview stereo techniques with image-based neural rendering for generalization, yielding impressive results, while suffering from three issues. First, occlusions often result in inconsistent feature matching. Then, they deliver distortions and artifacts in geometric discontinuities and locally sharp shapes due to their individual process of sampled points and rough feature aggregation. Third, their image-based representations experience severe degradations when source views are not near enough to the target view. To address challenges, we propose the first paradigm that constructs the generalizable neural field based on point-based rather than image-based rendering, which we call the Generalizable neural Point Field (GPF). Our approach explicitly models visibilities by geometric priors and augments them with neural features. We propose a novel nonuniform log sampling strategy to improve both rendering speed and reconstruction quality. Moreover, we present a learnable kernel spatially augmented with features for feature aggregations, mitigating distortions at places with drastically varying geometries. Besides, our representation can be easily manipulated. Experiments show that our model can deliver better geometries, view consistencies, and rendering quality than all counterparts and benchmarks on three datasets in both generalization and finetuning settings, preliminarily proving the potential of the new paradigm for generalizable NeRF.

翻译：本文提出了一种用于可泛化神经辐射场（NeRF）的新范式。以往的通用NeRF方法将多视图立体技术与基于图像的神经渲染相结合以实现泛化，虽取得了显著成果，但仍存在三个问题。第一，遮挡常导致特征匹配不一致。第二，由于对采样点进行独立处理且特征聚合粗糙，会在几何不连续区域与局部尖锐形状处产生畸变与伪影。第三，当源视图与目标视图相距较远时，基于图像的表示会出现严重退化。为解决这些问题，我们首次提出基于点渲染而非图像渲染构建可泛化神经场的范式，命名为通用神经点场（GPF）。该方法通过几何先验显式建模可见性，并用神经特征对其进行增强。我们提出了一种新颖的非均匀对数采样策略，以提高渲染速度与重建质量。此外，我们提出了一种可学习核函数，通过空间特征增强进行特征聚合，有效缓解了几何剧烈变化区域的畸变。同时，我们的表示易于操控。实验表明，在三个数据集的泛化与微调设置中，我们的模型在几何质量、视图一致性与渲染效果上均优于所有对比方法与基准，初步证明了该新范式在可泛化NeRF领域的潜力。