3D reconstruction of large scenes is a challenging problem due to the high-complexity nature of the solution space, in particular for generative neural networks. In contrast to traditional generative learned models which encode the full generative process into a neural network and can struggle with maintaining local details at the scene level, we introduce a new method that directly leverages scene geometry from the training database. First, we learn to synthesize an initial estimate for a 3D scene, constructed by retrieving a top-k set of volumetric chunks from the scene database. These candidates are then refined to a final scene generation with an attention-based refinement that can effectively select the most consistent set of geometry from the candidates and combine them together to create an output scene, facilitating transfer of coherent structures and local detail from train scene geometry. We demonstrate our neural scene reconstruction with a database for the tasks of 3D super resolution and surface reconstruction from sparse point clouds, showing that our approach enables generation of more coherent, accurate 3D scenes, improving on average by over 8% in IoU over state-of-the-art scene reconstruction.

翻译：3D 重建大场景是一个具有挑战性的问题,因为解决方案空间,特别是基因神经网络的复杂性很高。与传统的基因学模型不同,这些模型将整个基因变异过程编码成神经网络,并能够在现场一级努力维护当地细节,我们采用新方法,直接利用培训数据库中的场景几何学。首先,我们学习综合对3D场景的初步估计,从现场数据库中取回一组最强的体积块。然后,这些候选人被精细化成最后的场景一代,以关注为基础,从候选人中有效地选择一套最一致的几何方法,并把它们结合起来,创造出一个产出场景,便利从火车场景几何学中转移连贯的结构和当地细节。我们用一个数据库展示我们的神经场重建,用一个数据库来完成3D超分辨率的任务,从稀疏的云层进行地表重建,表明我们的方法能够产生更一致、准确的3D场景,在IoU中平均改善8%以上。