Recent advances in 3D scene representation and novel view synthesis have witnessed the rise of Neural Radiance Fields (NeRFs). Nevertheless, it is not trivial to exploit NeRF for the photorealistic 3D scene stylization task, which aims to generate visually consistent and photorealistic stylized scenes from novel views. Simply coupling NeRF with photorealistic style transfer (PST) will result in cross-view inconsistency and degradation of stylized view syntheses. Through a thorough analysis, we demonstrate that this non-trivial task can be simplified in a new light: When transforming the appearance representation of a pre-trained NeRF with Lipschitz mapping, the consistency and photorealism across source views will be seamlessly encoded into the syntheses. That motivates us to build a concise and flexible learning framework namely LipRF, which upgrades arbitrary 2D PST methods with Lipschitz mapping tailored for the 3D scene. Technically, LipRF first pre-trains a radiance field to reconstruct the 3D scene, and then emulates the style on each view by 2D PST as the prior to learn a Lipschitz network to stylize the pre-trained appearance. In view of that Lipschitz condition highly impacts the expressivity of the neural network, we devise an adaptive regularization to balance the reconstruction and stylization. A gradual gradient aggregation strategy is further introduced to optimize LipRF in a cost-efficient manner. We conduct extensive experiments to show the high quality and robust performance of LipRF on both photorealistic 3D stylization and object appearance editing.

翻译：近期三维场景表示与新视角合成领域的进展见证了神经辐射场（NeRF）的兴起。然而，利用NeRF实现真实感3D场景风格化任务并非易事——该任务旨在从新视角生成视觉一致且真实感强的风格化场景。简单地将NeRF与真实感风格迁移（PST）方法结合会导致跨视角不一致性以及风格化视角合成的质量退化。通过深入分析，我们证明这项棘手的任务可以从新角度得到简化：当使用Lipschitz映射对预训练NeRF的外观表示进行变换时，源视角间的一致性与真实感将被无缝编码到合成结果中。这促使我们构建了一个简洁且灵活的框架LipRF，该框架通过专为三维场景设计的Lipschitz映射，实现了对任意二维PST方法的升级。技术上，LipRF首先预训练辐射场以重建三维场景，随后利用二维PST对每个视角进行风格模拟作为先验，学习一个Lipschitz网络来对预训练外观进行风格化。鉴于Lipschitz条件对神经网络表达能力的关键影响，我们设计了自适应正则化来平衡重建与风格化任务。进一步引入渐进梯度聚合策略，以经济高效的方式优化LipRF。大量实验表明，LipRF在真实感三维风格化与物体外观编辑任务中均展现出高质量与鲁棒性能。