In recent years, 3D generation has made great strides in both academia and industry. However, generating 3D scenes from a single RGB image remains a significant challenge, as current approaches often struggle to ensure both object generation quality and scene coherence in multi-object scenarios. To overcome these limitations, we propose a novel three-stage framework for 3D scene generation with explicit geometric representations and high-quality textural details via single image-guided model generation and spatial layout optimization. Our method begins with an image instance segmentation and inpainting phase, which recovers missing details of occluded objects in the input images, thereby achieving complete generation of foreground 3D assets. Subsequently, our approach captures the spatial geometry of reference image by constructing pseudo-stereo viewpoint for camera parameter estimation and scene depth inference, while employing a model selection strategy to ensure optimal alignment between the 3D assets generated in the previous step and the input. Finally, through model parameterization and minimization of the Chamfer distance between point clouds in 3D and 2D space, our approach optimizes layout parameters to produce an explicit 3D scene representation that maintains precise alignment with input guidance image. Extensive experiments on multi-object scene image sets have demonstrated that our approach not only outperforms state-of-the-art methods in terms of geometric accuracy and texture fidelity of individual generated 3D models, but also has significant advantages in scene layout synthesis.

翻译：近年来，三维生成在学术界和工业界均取得了显著进展。然而，从单张RGB图像生成三维场景仍然是一个重大挑战，因为现有方法在多物体场景中往往难以同时保证物体生成质量与场景连贯性。为克服这些局限，我们提出了一种新颖的三阶段框架，通过单图像引导的模型生成与空间布局优化，实现具有显式几何表示和高质量纹理细节的三维场景生成。我们的方法始于图像实例分割与修复阶段，该阶段恢复输入图像中被遮挡物体的缺失细节，从而实现前景三维资产的完整生成。随后，本方法通过构建伪立体视点以估计相机参数并推断场景深度，从而捕捉参考图像的空间几何信息，同时采用模型选择策略以确保上一步生成的三维资产与输入图像达到最优对齐。最后，通过模型参数化并最小化三维与二维空间中点云之间的Chamfer距离，我们的方法优化布局参数以生成显式的三维场景表示，该表示与输入引导图像保持精确对齐。在多物体场景图像集上的大量实验表明，我们的方法不仅在生成单个三维模型的几何精度和纹理保真度方面优于现有先进方法，在场景布局合成方面也具有显著优势。