High-precision controllable remote sensing image generation is both meaningful and challenging. Existing diffusion models often produce low-fidelity images due to their inability to adequately capture morphological details, which may affect the robustness and reliability of object detection models. To enhance the accuracy and fidelity of generated objects in remote sensing, this paper proposes Object Fidelity Diffusion (OF-Diff), which effectively improves the fidelity of generated objects. Specifically, we are the first to extract the prior shapes of objects based on the layout for diffusion models in remote sensing. Then, we introduce a dual-branch diffusion model with diffusion consistency loss, which can generate high-fidelity remote sensing images without providing real images during the sampling phase. Furthermore, we introduce DDPO to fine-tune the diffusion process, making the generated remote sensing images more diverse and semantically consistent. Comprehensive experiments demonstrate that OF-Diff outperforms state-of-the-art methods in the remote sensing across key quality metrics. Notably, the performance of several polymorphic and small object classes shows significant improvement. For instance, the mAP increases by 8.3%, 7.7%, and 4.0% for airplanes, ships, and vehicles, respectively.
翻译:高精度可控遥感图像生成兼具重要意义与挑战性。现有扩散模型常因难以充分捕捉形态细节而生成低保真度图像,可能影响目标检测模型的鲁棒性与可靠性。为提升遥感场景中生成目标的精确度与保真度,本文提出目标保真扩散方法(OF-Diff),能有效提升生成目标的保真度。具体而言,我们首次基于布局信息为遥感扩散模型提取目标先验形状。随后,我们引入具有扩散一致性损失的双分支扩散模型,该模型可在采样阶段不提供真实图像的情况下生成高保真遥感图像。此外,我们采用DDPO对扩散过程进行微调,使生成的遥感图像更具多样性且保持语义一致性。综合实验表明,OF-Diff在遥感领域的关键质量指标上均优于现有先进方法。值得注意的是,多类多形态及小尺度目标类别的性能均呈现显著提升。例如,在飞机、船舶和车辆类别上,mAP指标分别提升了8.3%、7.7%和4.0%。