Neural rendering has demonstrated remarkable success in high-quality 3D neural reconstruction and novel view synthesis with dense input views and accurate poses. However, applying it to extremely sparse, unposed views in unbounded 360{\deg} scenes remains a challenging problem. In this paper, we propose a novel neural rendering framework to accomplish the unposed and extremely sparse-view 3D reconstruction in unbounded 360{\deg} scenes. To resolve the spatial ambiguity inherent in unbounded scenes with sparse input views, we propose a layered Gaussian-based representation to effectively model the scene with distinct spatial layers. By employing a dense stereo reconstruction model to recover coarse geometry, we introduce a layer-specific bootstrap optimization to refine the noise and fill occluded regions in the reconstruction. Furthermore, we propose an iterative fusion of reconstruction and generation alongside an uncertainty-aware training approach to facilitate mutual conditioning and enhancement between these two processes. Comprehensive experiments show that our approach outperforms existing state-of-the-art methods in terms of rendering quality and surface reconstruction accuracy. Project page: https://zju3dv.github.io/free360/

翻译：神经渲染在密集输入视角与精确位姿条件下的高质量三维神经重建与新视角合成方面取得了显著成功。然而，将其应用于无界360度场景中极稀疏且无位姿的输入视角，仍然是一个具有挑战性的问题。本文提出了一种新颖的神经渲染框架，旨在实现无界360度场景下的无位姿极稀疏视角三维重建。为缓解无界场景在稀疏输入视角下固有的空间模糊性问题，我们提出了一种基于分层高斯的表示方法，通过不同的空间层次有效建模场景。通过采用稠密立体重建模型恢复粗略几何，我们引入了层次特定的引导优化策略，以细化重建中的噪声并填补遮挡区域。此外，我们提出了一种重建与生成过程的迭代融合方法，并结合不确定性感知的训练策略，以促进这两个过程之间的相互条件约束与增强。综合实验表明，我们的方法在渲染质量与表面重建精度方面均优于现有的先进方法。项目页面：https://zju3dv.github.io/free360/