Multi-modal brain images from MRI scans are widely used in clinical diagnosis to provide complementary information from different modalities. However, obtaining fully paired multi-modal images in practice is challenging due to various factors, such as time, cost, and artifacts, resulting in modality-missing brain images. To address this problem, unsupervised multi-modal brain image translation has been extensively studied. Existing methods suffer from the problem of brain tumor deformation during translation, as they fail to focus on the tumor areas when translating the whole images. In this paper, we propose an unsupervised tumor-aware distillation teacher-student network called UTAD-Net, which is capable of perceiving and translating tumor areas precisely. Specifically, our model consists of two parts: a teacher network and a student network. The teacher network learns an end-to-end mapping from source to target modality using unpaired images and corresponding tumor masks first. Then, the translation knowledge is distilled into the student network, enabling it to generate more realistic tumor areas and whole images without masks. Experiments show that our model achieves competitive performance on both quantitative and qualitative evaluations of image quality compared with state-of-the-art methods. Furthermore, we demonstrate the effectiveness of the generated images on downstream segmentation tasks. Our code is available at https://github.com/scut-HC/UTAD-Net.
翻译:多模态MRI扫描脑图像在临床诊断中被广泛使用,以提供不同模态的互补信息。然而,由于时间、成本和伪影等多种因素,实践中难以获得完全配对的多模态图像,导致出现模态缺失的脑图像。为解决这一问题,无监督多模态脑图像翻译已被广泛研究。现有方法存在脑肿瘤在翻译过程中变形的问题,因为它们在翻译整图时未能聚焦于肿瘤区域。本文提出了一种名为UTAD-Net的无监督肿瘤感知蒸馏教师-学生网络,该网络能够精确感知和翻译肿瘤区域。具体而言,我们的模型由两部分组成:教师网络和学生网络。教师网络首先利用未配对图像及对应肿瘤掩膜学习从源模态到目标模态的端到端映射。随后,翻译知识被蒸馏到学生网络中,使其能够在无掩膜的情况下生成更逼真的肿瘤区域和整图。实验表明,与现有最优方法相比,我们的模型在图像质量的定量和定性评估中均取得了竞争性表现。此外,我们验证了生成图像在下游分割任务中的有效性。我们的代码开源在 https://github.com/scut-HC/UTAD-Net。