Neural Radiance Field (NeRF) has broken new ground in the novel view synthesis due to its simple concept and state-of-the-art quality. However, it suffers from severe performance degradation unless trained with a dense set of images with different camera poses, which hinders its practical applications. Although previous methods addressing this problem achieved promising results, they relied heavily on the additional training resources, which goes against the philosophy of sparse-input novel-view synthesis pursuing the training efficiency. In this work, we propose MixNeRF, an effective training strategy for novel view synthesis from sparse inputs by modeling a ray with a mixture density model. Our MixNeRF estimates the joint distribution of RGB colors along the ray samples by modeling it with mixture of distributions. We also propose a new task of ray depth estimation as a useful training objective, which is highly correlated with 3D scene geometry. Moreover, we remodel the colors with regenerated blending weights based on the estimated ray depth and further improves the robustness for colors and viewpoints. Our MixNeRF outperforms other state-of-the-art methods in various standard benchmarks with superior efficiency of training and inference.

翻译：神经辐射场（NeRF）因其简洁的概念与顶尖质量，在新视角合成领域开创了新局面。然而，该方法严重依赖密集的多视角图像集及不同相机位姿进行训练，否则性能会大幅退化，这阻碍了其实际应用。尽管先前解决该问题的方法取得了可喜成果，但它们严重依赖额外训练资源，这与追求训练效率的稀疏输入新视角合成理念相悖。本文提出MixNeRF——一种通过混合密度模型对射线进行建模的稀疏输入新视角合成高效训练策略。我们的MixNeRF通过混合分布对射线采样点沿线的RGB颜色联合分布进行建模。我们还提出了射线深度估计这一新任务作为有效的训练目标，该任务与三维场景几何高度相关。此外，我们基于估计的射线深度重构颜色混合权重，进一步提升了颜色与视角的鲁棒性。在多个标准基准测试中，我们的MixNeRF以更优的训练与推理效率超越了其他先进方法。