Generating complete 360-degree panoramas from narrow field of view images is ongoing research as omnidirectional RGB data is not readily available. Existing GAN-based approaches face some barriers to achieving higher quality output, and have poor generalization performance over different mask types. In this paper, we present our 360-degree indoor RGB panorama outpainting model using latent diffusion models (LDM), called IPO-LDM. We introduce a new bi-modal latent diffusion structure that utilizes both RGB and depth panoramic data during training, but works surprisingly well to outpaint normal depth-free RGB images during inference. We further propose a novel technique of introducing progressive camera rotations during each diffusion denoising step, which leads to substantial improvement in achieving panorama wraparound consistency. Results show that our IPO-LDM not only significantly outperforms state-of-the-art methods on RGB panorama outpainting, but can also produce multiple and diverse well-structured results for different types of masks.

翻译：从窄视场图像生成完整的360度全景图是一项持续的研究，因为全向RGB数据不易获得。现有的基于GAN的方法在获得更高质量输出方面面临一些障碍，且对不同遮罩类型的泛化性能较差。本文提出了一种使用潜扩散模型的360度室内RGB全景外推模型，称为IPO-LDM。我们引入了一种新的双模态潜扩散结构，在训练过程中同时利用RGB和深度全景数据，但在推理时却能出色地外推不含深度的普通RGB图像。此外，我们提出了一种新技术，即在每个扩散去噪步骤中引入渐进式相机旋转，从而显著提升了全景环绕一致性的实现效果。结果表明，我们的IPO-LDM不仅在RGB全景外推任务上显著优于现有最先进方法，还能针对不同类型的遮罩生成多个且多样化、结构良好的结果。