In the field of 3D content generation, single image scene reconstruction methods still struggle to simultaneously ensure the quality of individual assets and the coherence of the overall scene in complex environments, while texture editing techniques often fail to maintain both local continuity and multi-view consistency. In this paper, we propose a novel system ZeroScene, which leverages the prior knowledge of large vision models to accomplish both single image-to-3D scene reconstruction and texture editing in a zero-shot manner. ZeroScene extracts object-level 2D segmentation and depth information from input images to infer spatial relationships within the scene. It then jointly optimizes 3D and 2D projection losses of the point cloud to update object poses for precise scene alignment, ultimately constructing a coherent and complete 3D scene that encompasses both foreground and background. Moreover, ZeroScene supports texture editing of objects in the scene. By imposing constraints on the diffusion model and introducing a mask-guided progressive image generation strategy, we effectively maintain texture consistency across multiple viewpoints and further enhance the realism of rendered results through Physically Based Rendering (PBR) material estimation. Experimental results demonstrate that our framework not only ensures the geometric and appearance accuracy of generated assets, but also faithfully reconstructs scene layouts and produces highly detailed textures that closely align with text prompts.

翻译：在三维内容生成领域，单图像场景重建方法在复杂环境中仍难以同时保证单个资产的质量与整体场景的连贯性，而纹理编辑技术则往往无法兼顾局部连续性与多视角一致性。本文提出一种新颖的系统ZeroScene，该系统利用大规模视觉模型的先验知识，以零样本方式同时完成单图像到三维场景的重建与纹理编辑。ZeroScene从输入图像中提取物体级别的二维分割与深度信息，以推断场景内的空间关系。随后，通过联合优化点云的三维与二维投影损失来更新物体位姿，实现精确的场景对齐，最终构建出包含前景与背景的连贯且完整的三维场景。此外，ZeroScene支持对场景中物体的纹理编辑。通过对扩散模型施加约束并引入掩码引导的渐进式图像生成策略，我们有效保持了多视角间的纹理一致性，并通过基于物理的渲染（PBR）材质估计进一步提升了渲染结果的真实感。实验结果表明，我们的框架不仅保证了生成资产的几何与外观精度，还能忠实重建场景布局，并生成与文本提示高度吻合的精细纹理。