Editable high-fidelity 4D scenes are crucial for autonomous driving, as they can be applied to end-to-end training and closed-loop simulation. However, existing reconstruction methods are primarily limited to replicating observed scenes and lack the capability for diverse weather simulation. While image-level weather editing methods tend to introduce scene artifacts and offer poor controllability over the weather effects. To address these limitations, we propose \textbf{WeatherCity}, a novel framework for 4D urban scene reconstruction and weather editing. Specifically, we leverage a text-guided image editing model to achieve flexible editing of image weather backgrounds. To tackle the challenge of multi-weather modeling, we introduce a novel weather Gaussian representation based on shared scene features and dedicated weather-specific decoders. This representation is further enhanced with a content consistency optimization, ensuring coherent modeling across different weather conditions. Additionally, we design a physics-driven model that simulates dynamic weather effects through particles and motion patterns. Extensive experiments on multiple datasets and various scenes demonstrate that WeatherCity achieves flexible controllability, high fidelity, and temporal consistency in 4D reconstruction and weather editing. Our framework not only enables fine-grained control over weather conditions (e.g., light rain and heavy snow) but also supports object-level manipulation within the scene. Codes are released at https://github.com/IRMVLab/WeatherCity.
翻译:高保真可编辑的4D场景对自动驾驶至关重要,可应用于端到端训练及闭环仿真。然而现有重建方法主要局限于复现观测场景,缺乏多样天气模拟能力。基于图像的天气编辑方法易引入场景伪影,且对天气效果的可控性不足。为解决这些局限,我们提出\textbf{WeatherCity}——一种面向4D城市场景重建与天气编辑的新型框架。具体而言,我们利用文本引导的图像编辑模型实现图像天气背景的灵活编辑。为应对多天气建模挑战,我们基于共享场景特征与专用天气解码器,提出了一种新颖的天气高斯表示方法。该表示通过内容一致性优化进一步增强,确保不同天气条件间建模的连贯性。此外,我们设计了一个物理驱动模型,通过粒子与运动模式模拟动态天气效果。在多个数据集及各类场景上的大量实验表明,WeatherCity在4D重建与天气编辑中实现了灵活可控性、高保真度以及时间一致性。我们的框架不仅支持对天气条件(如小雨与大雪)的精细控制,还支持场景内对象级操作。代码已发布于https://github.com/IRMVLab/WeatherCity。