In this paper, we tackle the challenging task of Panoramic Image-to-Image translation (Pano-I2I) for the first time. This task is difficult due to the geometric distortion of panoramic images and the lack of a panoramic image dataset with diverse conditions, like weather or time. To address these challenges, we propose a panoramic distortion-aware I2I model that preserves the structure of the panoramic images while consistently translating their global style referenced from a pinhole image. To mitigate the distortion issue in naive 360 panorama translation, we adopt spherical positional embedding to our transformer encoders, introduce a distortion-free discriminator, and apply sphere-based rotation for augmentation and its ensemble. We also design a content encoder and a style encoder to be deformation-aware to deal with a large domain gap between panoramas and pinhole images, enabling us to work on diverse conditions of pinhole images. In addition, considering the large discrepancy between panoramas and pinhole images, our framework decouples the learning procedure of the panoramic reconstruction stage from the translation stage. We show distinct improvements over existing I2I models in translating the StreetLearn dataset in the daytime into diverse conditions. The code will be publicly available online for our community.

翻译：本文首次攻克了全景图像到图像翻译（Pano-I2I）这一具有挑战性的任务。该任务的难点在于全景图像的几何畸变，以及缺乏包含天气、时间等不同条件的全景图像数据集。为解决这些挑战，我们提出了一种全景畸变感知的I2I模型，该模型在保持全景图像结构的同时，能够基于针孔图像参考实现全局风格的一致性转换。为缓解朴素360度全景变换中的畸变问题，我们在Transformer编码器中采用了球面位置嵌入，引入无畸变鉴别器，并应用基于球面的旋转进行数据增强及集成。我们还设计了内容编码器和风格编码器，使其具备形变感知能力，以应对全景图与针孔图像之间巨大的域差距，从而能够处理不同条件下的针孔图像。此外，考虑到全景图与针孔图像之间的显著差异，我们的框架将全景重建阶段与翻译阶段的学习过程解耦。实验表明，在将StreetLearn数据集从白天场景翻译为多种条件时，我们的方法相比现有I2I模型取得了显著改进。代码将面向学术界公开。