Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

翻译：单张图像三维重建是一项重要但具有挑战性的任务，需要对自然世界有广泛认知。现有方法多在二维扩散模型指导下优化神经辐射场，但存在优化耗时长、三维结果不一致以及几何质量差等问题。本文提出一种新方法，以任意物体单张图像为输入，通过单次前馈推理即可生成完整360度带纹理的三维网格。给定单张图像后，我们首先利用视角条件二维扩散模型Zero123生成多视角图像，再将其提升至三维空间。针对传统重建方法难以处理不一致多视角预测的问题，我们基于可泛化的有符号距离场神经表面重建方法构建三维重建模块，并提出多项关键训练策略以实现360度网格重建。无需高昂优化成本，本方法重建三维形状的时间显著少于现有方法。此外，本方法能生成更优的几何形状、更一致的三维结果，且更贴合输入图像。我们在合成数据和真实场景图像上评估了本方法，证明其在网格质量与运行时间方面的优越性。同时，本方法可通过集成现成文本到图像扩散模型，无缝支持文本到三维任务。