In recent years, there has been increasing interest in applying stylization on 3D scenes from a reference style image, in particular onto neural radiance fields (NeRF). While performing stylization directly on NeRF guarantees appearance consistency over arbitrary novel views, it is a challenging problem to guide the transfer of patterns from the style image onto different parts of the NeRF scene. In this work, we propose a stylization framework for NeRF based on local style transfer. In particular, we use a hash-grid encoding to learn the embedding of the appearance and geometry components, and show that the mapping defined by the hash table allows us to control the stylization to a certain extent. Stylization is then achieved by optimizing the appearance branch while keeping the geometry branch fixed. To support local style transfer, we propose a new loss function that utilizes a segmentation network and bipartite matching to establish region correspondences between the style image and the content images obtained from volume rendering. Our experiments show that our method yields plausible stylization results with novel view synthesis while having flexible controllability via manipulating and customizing the region correspondences.

翻译：近年来，从参考风格图像对三维场景（特别是神经辐射场NeRF）进行风格化引起了广泛关注。虽然直接在NeRF上进行风格化能保证任意新视角下的外观一致性，但如何引导风格图像的图案迁移到NeRF场景的不同部分仍是一个难题。本文提出了一种基于局部风格迁移的NeRF风格化框架。具体而言，我们采用哈希网格编码学习外观和几何组件的嵌入，并证明哈希表定义的映射能在一定程度上对风格化过程进行控制。通过固定几何分支并优化外观分支实现风格化。为了支持局部风格迁移，我们提出了一种新的损失函数，利用分割网络和二分图匹配建立风格图像与体渲染得到的内容图像之间的区域对应关系。实验结果表明，该方法在实现新视角合成时能生成合理的风格化结果，同时通过操控和定制区域对应关系具有灵活的可控性。