A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.
翻译:神经辐射场(NeRF)编码了场景三维几何与外观之间的特定关系。我们在此探究一个问题:能否以语义有意义的方式将源NeRF的外观迁移至目标三维几何上,使得生成的新NeRF保留目标几何结构,同时具有与源NeRF类比的外观。为此,我们将经典的图像类比方法从二维图像推广至NeRF。我们利用大规模预训练二维图像模型中的语义特征,通过语义亲和性进行对应关系迁移,从而实现多视角一致的外观迁移。我们的方法支持探索三维几何与外观的混合匹配产品空间。实验表明,本方法优于传统风格化方法,且绝大多数用户更倾向于选择我们的方法而非多种典型基线方法。