We present Instant Neural Radiance Fields Stylization, a novel approach for multi-view image stylization for the 3D scene. Our approach models a neural radiance field based on neural graphics primitives, which use a hash table-based position encoder for position embedding. We split the position encoder into two parts, the content and style sub-branches, and train the network for normal novel view image synthesis with the content and style targets. In the inference stage, we execute AdaIN to the output features of the position encoder, with content and style voxel grid features as reference. With the adjusted features, the stylization of novel view images could be obtained. Our method extends the style target from style images to image sets of scenes and does not require additional network training for stylization. Given a set of images of 3D scenes and a style target(a style image or another set of 3D scenes), our method can generate stylized novel views with a consistent appearance at various view angles in less than 10 minutes on modern GPU hardware. Extensive experimental results demonstrate the validity and superiority of our method.
翻译:我们提出了一种名为即时神经辐射场风格化的新颖方法,用于三维场景的多视角图像风格化。该方法基于神经图形基元构建神经辐射场,并采用哈希表式位置编码器进行位置嵌入。我们将位置编码器分为内容与风格两个子分支,利用内容和风格目标训练网络,实现标准的新视角图像合成。在推理阶段,我们以内容和风格体素网格特征为参考,对位置编码器的输出特征执行自适应实例归一化(AdaIN)。通过调整后的特征,即可获得新视角图像的风格化效果。本方法将风格目标从风格图像扩展至场景图像集,且无需针对风格化进行额外网络训练。给定一组三维场景图像与一个风格目标(风格图像或另一三维场景集),我们的方法能在现代GPU硬件上于10分钟内生成具有一致外观的多视角风格化新视图。大量实验结果表明了本方法的有效性与优越性。