Existing inverse rendering combined with neural rendering methods~\cite{zhang2021physg, zhang2022modeling} can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based~\cite{mildenhall2020nerf} neural rendering method and can extend its application to room-scale scenes. Since intrinsic decomposition is a fundamentally under-constrained inverse problem, we propose a novel distance-aware point sampling and adaptive reflectance iterative clustering optimization method, which enables IntrinsicNeRF with traditional intrinsic decomposition constraints to be trained in an unsupervised manner, resulting in temporally consistent intrinsic decomposition results. To cope with the problem that different adjacent instances of similar reflectance in a scene are incorrectly clustered together, we further propose a hierarchical clustering method with coarse-to-fine optimization to obtain a fast hierarchical indexing representation. It supports compelling real-time augmented applications such as recoloring and illumination variation. Extensive experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results and high-fidelity novel view synthesis even for challenging sequences. Project page: https://zju3dv.github.io/intrinsic_nerf.

翻译：现有的逆渲染结合神经渲染方法~\cite{zhang2021physg, zhang2022modeling}仅能在物体级场景中实现可编辑的新视角合成，而本文提出内禀神经辐射场（IntrinsicNeRF），将内禀分解引入基于NeRF~\cite{mildenhall2020nerf}的神经渲染方法中，并将其应用扩展到房间级场景。由于内禀分解本质上是一个欠约束的逆问题，我们提出一种新颖的距离感知点采样与自适应反射率迭代聚类优化方法，使得融合传统内禀分解约束的IntrinsicNeRF能以无监督方式训练，从而获得时间一致的内禀分解结果。针对场景中不同邻近实例因相似反射率而被错误聚类的问题，我们进一步提出一种结合粗到细优化的层次化聚类方法，以获取快速的层次化索引表示。该表示支持重着色与光照变化等令人瞩目的实时增强应用。在物体级/房间级场景以及合成/真实数据上的大量实验和编辑示例表明，即使对于具有挑战性的序列，我们也能获得一致的内禀分解结果和高保真的新视角合成。项目页面：https://zju3dv.github.io/intrinsic_nerf。