Recently, neural radiance field (NeRF) has shown remarkable performance in novel view synthesis and 3D reconstruction. However, it still requires abundant high-quality images, limiting its applicability in real-world scenarios. To overcome this limitation, recent works have focused on training NeRF only with sparse viewpoints by giving additional regularizations, often called few-shot NeRF. We observe that due to the under-constrained nature of the task, solely using additional regularization is not enough to prevent the model from overfitting to sparse viewpoints. In this paper, we propose a novel framework, dubbed Self-Evolving Neural Radiance Fields (SE-NeRF), that applies a self-training framework to NeRF to address these problems. We formulate few-shot NeRF into a teacher-student framework to guide the network to learn a more robust representation of the scene by training the student with additional pseudo labels generated from the teacher. By distilling ray-level pseudo labels using distinct distillation schemes for reliable and unreliable rays obtained with our novel reliability estimation method, we enable NeRF to learn a more accurate and robust geometry of the 3D scene. We show and evaluate that applying our self-training framework to existing models improves the quality of the rendered images and achieves state-of-the-art performance in multiple settings.

翻译：最近，神经辐射场（NeRF）在新视角合成与三维重建领域展现出卓越性能。然而，该方法仍依赖大量高质量图像，限制了其在真实场景中的适用性。为突破这一局限，近期研究通过引入额外正则化项，致力于仅利用稀疏视角训练NeRF（常被称为少样本NeRF）。我们观察到，受限于任务本身的欠约束特性，仅依赖额外正则化不足以防止模型对稀疏视角的过拟合。本文提出一种名为自演化神经辐射场（SE-NeRF）的新型框架，通过向NeRF引入自训练机制来解决上述问题。我们将少样本NeRF建模为教师-学生框架：教师网络生成额外伪标签指导学生网络训练，从而提升场景表征的鲁棒性。通过基于我们提出的可靠性估计方法区分可靠与不可靠光线，并采用差异化蒸馏方案对光线级伪标签进行提纯，使NeRF能够学习到更精确、更鲁棒的三维场景几何结构。实验证明，将我们的自训练框架应用于现有模型，可显著提升渲染图像质量，并在多种设定下达到最先进性能。