Neural Radiance Fields (NeRF) have demonstrated impressive potential in synthesizing novel views from dense input, however, their effectiveness is challenged when dealing with sparse input. Existing approaches that incorporate additional depth or semantic supervision can alleviate this issue to an extent. However, the process of supervision collection is not only costly but also potentially inaccurate, leading to poor performance and generalization ability in diverse scenarios. In our work, we introduce a novel model: the Collaborative Neural Radiance Fields (ColNeRF) designed to work with sparse input. The collaboration in ColNeRF includes both the cooperation between sparse input images and the cooperation between the output of the neural radiation field. Through this, we construct a novel collaborative module that aligns information from various views and meanwhile imposes self-supervised constraints to ensure multi-view consistency in both geometry and appearance. A Collaborative Cross-View Volume Integration module (CCVI) is proposed to capture complex occlusions and implicitly infer the spatial location of objects. Moreover, we introduce self-supervision of target rays projected in multiple directions to ensure geometric and color consistency in adjacent regions. Benefiting from the collaboration at the input and output ends, ColNeRF is capable of capturing richer and more generalized scene representation, thereby facilitating higher-quality results of the novel view synthesis. Extensive experiments demonstrate that ColNeRF outperforms state-of-the-art sparse input generalizable NeRF methods. Furthermore, our approach exhibits superiority in fine-tuning towards adapting to new scenes, achieving competitive performance compared to per-scene optimized NeRF-based methods while significantly reducing computational costs. Our code is available at: https://github.com/eezkni/ColNeRF.

翻译：神经辐射场（NeRF）在密集输入的新视角合成中展现出显著潜力，然而当处理稀疏输入时其有效性面临挑战。现有方法通过引入额外深度或语义监督可在一定程度上缓解该问题，但监督数据的采集过程不仅代价高昂且可能不准确，导致在多样场景中性能与泛化能力欠佳。本文提出一种新颖模型——协同神经辐射场（ColNeRF），旨在处理稀疏输入。ColNeRF的协同包括稀疏输入图像间的协同以及神经辐射场输出间的协同。通过此机制，我们构建了一个新颖的协同模块，该模块不仅能对齐来自不同视角的信息，同时施加自监督约束以确保几何与外观的多视角一致性。此外，我们提出协同跨视角体积整合模块（CCVI）以捕获复杂遮挡并隐式推断物体的空间位置。更进一步，我们引入目标射线在多个方向上的投影自监督机制，确保相邻区域的几何与色彩一致性。受益于输入与输出端的协同设计，ColNeRF能够捕获更丰富且更具泛化性的场景表征，从而促进更高质量的新视角合成。大量实验表明，ColNeRF在性能上优于当前最先进的稀疏输入可泛化NeRF方法。此外，本方法在面向新场景的微调中展现出优越性，在与逐场景优化NeRF方法相比时，能以显著降低的计算成本达到竞争性性能。代码已开源：https://github.com/eezkni/ColNeRF。