Neural Radiance Fields (NeRF) have demonstrated impressive potential in synthesizing novel views from dense input, however, their effectiveness is challenged when dealing with sparse input. Existing approaches that incorporate additional depth or semantic supervision can alleviate this issue to an extent. However, the process of supervision collection is not only costly but also potentially inaccurate, leading to poor performance and generalization ability in diverse scenarios. In our work, we introduce a novel model: the Collaborative Neural Radiance Fields (ColNeRF) designed to work with sparse input. The collaboration in ColNeRF includes both the cooperation between sparse input images and the cooperation between the output of the neural radiation field. Through this, we construct a novel collaborative module that aligns information from various views and meanwhile imposes self-supervised constraints to ensure multi-view consistency in both geometry and appearance. A Collaborative Cross-View Volume Integration module (CCVI) is proposed to capture complex occlusions and implicitly infer the spatial location of objects. Moreover, we introduce self-supervision of target rays projected in multiple directions to ensure geometric and color consistency in adjacent regions. Benefiting from the collaboration at the input and output ends, ColNeRF is capable of capturing richer and more generalized scene representation, thereby facilitating higher-quality results of the novel view synthesis. Extensive experiments demonstrate that ColNeRF outperforms state-of-the-art sparse input generalizable NeRF methods. Furthermore, our approach exhibits superiority in fine-tuning towards adapting to new scenes, achieving competitive performance compared to per-scene optimized NeRF-based methods while significantly reducing computational costs. Our code is available at: https://github.com/eezkni/ColNeRF.

翻译：神经辐射场（NeRF）在密集输入下的新视角合成中展现出巨大潜力，然而其在处理稀疏输入时面临有效性挑战。现有方法通过引入额外深度或语义监督可在一定程度上缓解此问题，但监督数据的采集过程不仅成本高昂，还可能存在不准确性，导致模型在多样化场景中性能及泛化能力不佳。本文提出一种新型模型——协作式神经辐射场（ColNeRF），专为稀疏输入场景设计。ColNeRF的协作机制包含两方面：稀疏输入图像间的协同，以及神经辐射场输出间的协同。基于此，我们构建了一个新型协作模块，既能对齐多视角信息，又能通过自监督约束确保几何与外观的多视角一致性。此外，我们提出跨视角体素集成协作模块（CCVI），用于捕获复杂遮挡并隐式推断物体的空间位置。同时，我们引入多方向投影目标射线的自监督机制，以确保相邻区域的几何与颜色一致性。得益于输入与输出端的协同作用，ColNeRF能够捕获更丰富、更具泛化性的场景表征，从而提升新视角合成的质量。大量实验表明，ColNeRF优于当前最先进的稀疏输入可泛化NeRF方法。此外，本方法在适配新场景的微调过程中展现出优越性，相较于逐场景优化的NeRF方法，在显著降低计算成本的同时达到了竞争性性能。代码已开源于：https://github.com/eezkni/ColNeRF。