Unpaired image-to-image translation of retinal images can efficiently increase the training dataset for deep-learning-based multi-modal retinal registration methods. Our method integrates a vessel segmentation network into the image-to-image translation task by extending the CycleGAN framework. The segmentation network is inserted prior to a UNet vision transformer generator network and serves as a shared representation between both domains. We reformulate the original identity loss to learn the direct mapping between the vessel segmentation and the real image. Additionally, we add a segmentation loss term to ensure shared vessel locations between fake and real images. In the experiments, our method shows a visually realistic look and preserves the vessel structures, which is a prerequisite for generating multi-modal training data for image registration.

翻译：无配对视网膜图像到图像的翻译可有效扩充基于深度学习的多模态视网膜配准方法的训练数据集。我们的方法通过扩展CycleGAN框架，将血管分割网络集成到图像到图像翻译任务中。该分割网络被插入到UNet视觉变换器生成器网络之前，并作为两个域之间的共享表示。我们重新表述了原始身份损失函数，以学习血管分割与真实图像之间的直接映射。此外，我们添加了分割损失项，以确保假图像和真实图像之间共享血管位置。在实验中，我们的方法呈现出视觉上真实的外观，并保留了血管结构，这是为图像配准生成多模态训练数据的先决条件。