We present Instant Neural Radiance Fields Stylization, a novel approach for multi-view image stylization for the 3D scene. Our approach models a neural radiance field based on neural graphics primitives, which use a hash table-based position encoder for position embedding. We split the position encoder into two parts, the content and style sub-branches, and train the network for normal novel view image synthesis with the content and style targets. In the inference stage, we execute AdaIN to the output features of the position encoder, with content and style voxel grid features as reference. With the adjusted features, the stylization of novel view images could be obtained. Our method extends the style target from style images to image sets of scenes and does not require additional network training for stylization. Given a set of images of 3D scenes and a style target(a style image or another set of 3D scenes), our method can generate stylized novel views with a consistent appearance at various view angles in less than 10 minutes on modern GPU hardware. Extensive experimental results demonstrate the validity and superiority of our method.
翻译:我们提出了一种新颖的多视角图像风格化方法——即时神经辐射场风格化,用于三维场景。该方法基于神经图形基元构建神经辐射场模型,采用基于哈希表的位置编码器进行位置嵌入。我们将位置编码器拆分为内容与风格两个子分支,并利用内容和风格目标训练网络以合成正常新视角图像。在推理阶段,我们对位置编码器的输出特征执行AdaIN操作,以内容和风格体素网格特征作为参考。通过调整后的特征,即可获得新视角图像的风格化结果。本方法将风格目标从风格图像扩展至场景图像集,且无需为风格化进行额外的网络训练。给定一组三维场景图像及风格目标(风格图像或另一组三维场景),本方法可在现代GPU硬件上于10分钟内生成具有多视角外观一致性的风格化新视角图像。大量实验结果验证了本方法的有效性与优越性。