Creating high-fidelity 3D models of indoor environments is essential for applications in design, virtual reality, and robotics. However, manual 3D modeling remains time-consuming and labor-intensive. While recent advances in generative AI have enabled automated scene synthesis, existing methods often face challenges in balancing visual quality, diversity, semantic consistency, and user control. A major bottleneck is the lack of a large-scale, high-quality dataset tailored to this task. To address this gap, we introduce a comprehensive synthetic dataset, featuring 12,328 structured annotated scenes with 57,431 rooms, and 4.7M photorealistic 2D renderings. Leveraging this dataset, we present SpatialGen, a novel multi-view multi-modal diffusion model that generates realistic and semantically consistent 3D indoor scenes. Given a 3D layout and a reference image (derived from a text prompt), our model synthesizes appearance (color image), geometry (scene coordinate map), and semantic (semantic segmentation map) from arbitrary viewpoints, while preserving spatial consistency across modalities. SpatialGen consistently generates superior results to previous methods in our experiments. We are open-sourcing our data and models to empower the community and advance the field of indoor scene understanding and generation.

翻译：创建高保真的室内环境三维模型对于设计、虚拟现实和机器人技术等应用至关重要。然而，手动三维建模仍然耗时且劳动密集。尽管生成式人工智能的最新进展已实现自动化场景合成，但现有方法在平衡视觉质量、多样性、语义一致性和用户控制方面常常面临挑战。一个主要瓶颈是缺乏针对此任务的大规模高质量数据集。为填补这一空白，我们引入了一个全面的合成数据集，包含12,328个结构化标注场景、57,431个房间以及470万张逼真的二维渲染图像。利用该数据集，我们提出了SpatialGen，一种新颖的多视图多模态扩散模型，用于生成逼真且语义一致的三维室内场景。给定一个三维布局和一张参考图像（源自文本提示），我们的模型能够从任意视点合成外观（彩色图像）、几何（场景坐标图）和语义（语义分割图），同时保持跨模态的空间一致性。在我们的实验中，SpatialGen持续生成优于先前方法的结果。我们将开源我们的数据和模型，以赋能研究社区并推动室内场景理解与生成领域的发展。