Generalizable neural implicit surface reconstruction aims to obtain an accurate underlying geometry given a limited number of multi-view images from unseen scenes. However, existing methods select only informative and relevant views using predefined scores for training and testing phases. This constraint renders the model impractical in real-world scenarios, where the availability of favorable combinations cannot always be ensured. We introduce and validate a view-combination score to indicate the effectiveness of the input view combination. We observe that previous methods output degenerate solutions under arbitrary and unfavorable sets. Building upon this finding, we propose UFORecon, a robust view-combination generalizable surface reconstruction framework. To achieve this, we apply cross-view matching transformers to model interactions between source images and build correlation frustums to capture global correlations. Additionally, we explicitly encode pairwise feature similarities as view-consistent priors. Our proposed framework significantly outperforms previous methods in terms of view-combination generalizability and also in the conventional generalizable protocol trained with favorable view-combinations. The code is available at https://github.com/Youngju-Na/UFORecon.
翻译:泛化神经隐式表面重建旨在从有限数量的未见场景多视角图像中获取精确的底层几何结构。然而,现有方法仅通过预定义分数选择具有信息量和相关性的视图用于训练和测试阶段。这一约束使得模型在现实场景中不实用,因为有利视角组合的可用性无法始终得到保证。我们提出并验证了一种视角组合分数,用于指示输入视角组合的有效性。我们观察到,在任意不利视图集下,先前方法会输出退化解。基于这一发现,我们提出UFORecon,一种鲁棒的视角组合泛化表面重建框架。为实现该目标,我们应用跨视图匹配变换器建模源图像间的交互,并构建相关截锥体以捕获全局相关性。此外,我们显式编码成对特征相似性作为视图一致性先验。我们的框架在视角组合泛化性方面显著优于先前方法,同时在传统使用有利视角组合训练的泛化协议中也表现更佳。代码见https://github.com/Youngju-Na/UFORecon。