Current 3D stylization methods often assume static scenes, which violates the dynamic nature of our real world. To address this limitation, we present S-DyRF, a reference-based spatio-temporal stylization method for dynamic neural radiance fields. However, stylizing dynamic 3D scenes is inherently challenging due to the limited availability of stylized reference images along the temporal axis. Our key insight lies in introducing additional temporal cues besides the provided reference. To this end, we generate temporal pseudo-references from the given stylized reference. These pseudo-references facilitate the propagation of style information from the reference to the entire dynamic 3D scene. For coarse style transfer, we enforce novel views and times to mimic the style details present in pseudo-references at the feature level. To preserve high-frequency details, we create a collection of stylized temporal pseudo-rays from temporal pseudo-references. These pseudo-rays serve as detailed and explicit stylization guidance for achieving fine style transfer. Experiments on both synthetic and real-world datasets demonstrate that our method yields plausible stylized results of space-time view synthesis on dynamic 3D scenes.
翻译:当前的3D风格化方法通常假设场景是静态的,这与现实世界的动态特性相违背。为解决这一局限,我们提出S-DyRF——一种面向动态神经辐射场的、基于参考的时空风格化方法。然而,由于时序轴上可用的风格化参考图像有限,对动态3D场景进行风格化本身极具挑战性。我们的关键洞察在于,除了提供的参考外,还需引入额外的时序线索。为此,我们从给定的风格化参考中生成时序伪参考,这些伪参考有助于将风格信息从参考传播至整个动态3D场景。在粗粒度风格迁移阶段,我们强制新视角与新时刻在特征层面模仿伪参考中的风格细节;为保留高频细节,我们基于时序伪参考构建一组风格化时序伪光线,这些伪光线作为精细且显式的风格化指引,实现细粒度风格迁移。在合成和真实数据集上的实验表明,本方法能在动态3D场景的时空视角合成中生成合理的风格化结果。