Neural Radiance Field (NeRF) has broken new ground in the novel view synthesis due to its simple concept and state-of-the-art quality. However, it suffers from severe performance degradation unless trained with a dense set of images with different camera poses, which hinders its practical applications. Although previous methods addressing this problem achieved promising results, they relied heavily on the additional training resources, which goes against the philosophy of sparse-input novel-view synthesis pursuing the training efficiency. In this work, we propose MixNeRF, an effective training strategy for novel view synthesis from sparse inputs by modeling a ray with a mixture density model. Our MixNeRF estimates the joint distribution of RGB colors along the ray samples by modeling it with mixture of distributions. We also propose a new task of ray depth estimation as a useful training objective, which is highly correlated with 3D scene geometry. Moreover, we remodel the colors with regenerated blending weights based on the estimated ray depth and further improves the robustness for colors and viewpoints. Our MixNeRF outperforms other state-of-the-art methods in various standard benchmarks with superior efficiency of training and inference.

翻译：神经辐射场（NeRF）凭借其简洁的概念和领先的质量，在新型视角合成领域开创了新方向。然而，该方法需要密集的不同相机位姿图像进行训练，否则将出现严重的性能退化，这阻碍了其实际应用。尽管解决该问题的先前方法取得了可喜成果，但严重依赖额外训练资源，违背了稀疏输入新型视图合成追求训练效率的核心理念。本文提出MixNeRF——一种通过光线混合密度建模实现稀疏输入新型视角合成的有效训练策略。该模型通过混合分布建模沿光线采样的RGB颜色联合分布，同时提出光线深度估计这一与三维场景几何高度相关的新训练目标。基于估计的光线深度，我们重新生成混合权重对颜色进行重构，进一步提升了颜色和视角的鲁棒性。在多个标准基准测试中，我们的MixNeRF以卓越的训练和推理效率超越了其他先进方法。