Referenced-based scene stylization that edits the appearance based on a content-aligned reference image is an emerging research area. Starting with a pretrained neural radiance field (NeRF), existing methods typically learn a novel appearance that matches the given style. Despite their effectiveness, they inherently suffer from time-consuming volume rendering, and thus are impractical for many real-time applications. In this work, we propose ReGS, which adapts 3D Gaussian Splatting (3DGS) for reference-based stylization to enable real-time stylized view synthesis. Editing the appearance of a pretrained 3DGS is challenging as it uses discrete Gaussians as 3D representation, which tightly bind appearance with geometry. Simply optimizing the appearance as prior methods do is often insufficient for modeling continuous textures in the given reference image. To address this challenge, we propose a novel texture-guided control mechanism that adaptively adjusts local responsible Gaussians to a new geometric arrangement, serving for desired texture details. The proposed process is guided by texture clues for effective appearance editing, and regularized by scene depth for preserving original geometric structure. With these novel designs, we show ReGs can produce state-of-the-art stylization results that respect the reference texture while embracing real-time rendering speed for free-view navigation.

翻译：基于参考的场景风格化是一种新兴的研究方向，它根据内容对齐的参考图像来编辑场景外观。现有方法通常从预训练的神经辐射场（NeRF）出发，学习与给定风格匹配的新外观。尽管这些方法有效，但它们本质上受限于耗时的体渲染过程，因此在许多实时应用中并不实用。在本工作中，我们提出了ReGS，该方法将3D高斯溅射（3DGS）适配于基于参考的风格化任务，以实现实时的风格化视图合成。编辑预训练3DGS的外观具有挑战性，因为它使用离散高斯函数作为3D表示，将外观与几何结构紧密绑定。若仅如先前方法那样优化外观，通常不足以建模给定参考图像中的连续纹理。为应对这一挑战，我们提出了一种新颖的纹理引导控制机制，该机制自适应地将局部相关高斯函数调整至新的几何排布，以服务于所需的纹理细节。该过程由纹理线索引导以实现有效的外观编辑，并通过场景深度进行正则化以保持原始几何结构。凭借这些新颖设计，我们证明ReGS能够生成最先进的风格化结果，这些结果既忠实于参考纹理，又具备实时渲染速度，支持自由视角导航。