Mesh models have become increasingly accessible for numerous cities; however, the lack of realistic textures restricts their application in virtual urban navigation and autonomous driving. To address this, this paper proposes MeSS (Meshbased Scene Synthesis) for generating high-quality, styleconsistent outdoor scenes with city mesh models serving as the geometric prior. While image and video diffusion models can leverage spatial layouts (such as depth maps or HD maps) as control conditions to generate street-level perspective views, they are not directly applicable to 3D scene generation. Video diffusion models excel at synthesizing consistent view sequences that depict scenes but often struggle to adhere to predefined camera paths or align accurately with rendered control videos. In contrast, image diffusion models, though unable to guarantee cross-view visual consistency, can produce more geometry-aligned results when combined with ControlNet. Building on this insight, our approach enhances image diffusion models by improving cross-view consistency. The pipeline comprises three key stages: first, we generate geometrically consistent sparse views using Cascaded Outpainting ControlNets; second, we propagate denser intermediate views via a component dubbed AGInpaint; and third, we globally eliminate visual inconsistencies (e.g., varying exposure) using the GCAlign module. Concurrently with generation, a 3D Gaussian Splatting (3DGS) scene is reconstructed by initializing Gaussian balls on the mesh surface. Our method outperforms existing approaches in both geometric alignment and generation quality. Once synthesized, the scene can be rendered in diverse styles through relighting and style transfer techniques. project page: https://albertchen98.github.io/mess/

翻译：城市网格模型已日益普及，然而缺乏真实纹理限制了其在虚拟城市导航与自动驾驶中的应用。为此，本文提出MeSS（基于网格的场景合成）方法，以城市网格模型作为几何先验，生成高质量、风格一致的户外场景。尽管图像与视频扩散模型可利用空间布局（如深度图或高精地图）作为控制条件生成街景级透视图，但它们无法直接应用于三维场景生成。视频扩散模型擅长合成描述场景的一致性视角序列，但往往难以遵循预设相机路径或与渲染的控制视频精确对齐。相比之下，图像扩散模型虽无法保证跨视角视觉一致性，但与ControlNet结合时能产生更贴合几何结构的结果。基于此洞见，我们的方法通过提升跨视角一致性来增强图像扩散模型。该流程包含三个关键阶段：首先，我们使用级联外绘ControlNet生成几何一致的稀疏视角；其次，通过名为AGInpaint的组件传播更密集的中间视角；最后，利用GCAlign模块全局消除视觉不一致性（如曝光差异）。在生成过程中，我们通过在网格表面初始化高斯球体同步重建三维高斯泼溅场景。本方法在几何对齐与生成质量方面均优于现有方法。合成后的场景可通过重光照与风格迁移技术以多样风格进行渲染。项目页面：https://albertchen98.github.io/mess/