The explicit neural radiance field (NeRF) has gained considerable interest for its efficient training and fast inference capabilities, making it a promising direction such as virtual reality and gaming. In particular, PlenOctree (POT)[1], an explicit hierarchical multi-scale octree representation, has emerged as a structural and influential framework. However, POT's fixed structure for direct optimization is sub-optimal as the scene complexity evolves continuously with updates to cached color and density, necessitating refining the sampling distribution to capture signal complexity accordingly. To address this issue, we propose the dynamic PlenOctree DOT, which adaptively refines the sample distribution to adjust to changing scene complexity. Specifically, DOT proposes a concise yet novel hierarchical feature fusion strategy during the iterative rendering process. Firstly, it identifies the regions of interest through training signals to ensure adaptive and efficient refinement. Next, rather than directly filtering out valueless nodes, DOT introduces the sampling and pruning operations for octrees to aggregate features, enabling rapid parameter learning. Compared with POT, our DOT outperforms it by enhancing visual quality, reducing over $55.15$/$68.84\%$ parameters, and providing 1.7/1.9 times FPS for NeRF-synthetic and Tanks $\&$ Temples, respectively. Project homepage:https://vlislab22.github.io/DOT. [1] Yu, Alex, et al. "Plenoctrees for real-time rendering of neural radiance fields." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
翻译:显式神经辐射场(NeRF)因其高效训练和快速推理能力而备受关注,成为虚拟现实和游戏等领域极具前景的方向。其中,基于显式分层多尺度八叉树结构的PlenOctree(POT)[1] 作为具有结构性和影响力的框架应运而生。然而,POT用于直接优化的固定结构在场景复杂度随缓存颜色与密度持续变化时表现次优,需要优化采样分布以相应捕获信号复杂度。针对该问题,我们提出动态多叉树(DOT),通过自适应优化采样分布来适应变化的场景复杂度。具体而言,DOT在迭代渲染过程中提出一种简洁新颖的分层特征融合策略:首先通过训练信号识别感兴趣区域,确保自适应高效优化;其次,不同于直接剔除无价值节点,DOT引入八叉树的采样与剪枝操作来聚合特征,实现快速参数学习。与POT相比,我们的DOT在视觉质量提升、参数量减少超55.15%/68.84%以及NeRF-synthetic和Tanks & Temples数据集上分别实现1.7倍/1.9倍帧率提升方面均表现更优。项目主页:https://vlislab22.github.io/DOT。 [1] Yu, Alex, 等. "Plenoctrees for real-time rendering of neural radiance fields." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.