Medical image segmentation methods normally perform poorly when there is a domain shift between training and testing data. Unsupervised Domain Adaptation (UDA) addresses the domain shift problem by training the model using both labeled data from the source domain and unlabeled data from the target domain. Source-Free UDA (SFUDA) was recently proposed for UDA without requiring the source data during the adaptation, due to data privacy or data transmission issues, which normally adapts the pre-trained deep model in the testing stage. However, in real clinical scenarios of medical image segmentation, the trained model is normally frozen in the testing stage. In this paper, we propose Fourier Visual Prompting (FVP) for SFUDA of medical image segmentation. Inspired by prompting learning in natural language processing, FVP steers the frozen pre-trained model to perform well in the target domain by adding a visual prompt to the input target data. In FVP, the visual prompt is parameterized using only a small amount of low-frequency learnable parameters in the input frequency space, and is learned by minimizing the segmentation loss between the predicted segmentation of the prompted target image and reliable pseudo segmentation label of the target image under the frozen model. To our knowledge, FVP is the first work to apply visual prompts to SFUDA for medical image segmentation. The proposed FVP is validated using three public datasets, and experiments demonstrate that FVP yields better segmentation results, compared with various existing methods.
翻译:医学图像分割方法在训练数据与测试数据之间存在域偏移时通常表现不佳。无监督域适应(UDA)通过使用来自源域的标注数据和目标域的无标注数据训练模型来解决域偏移问题。由于数据隐私或数据传输问题,最近提出了无源无监督域适应(SFUDA),该方法在适应过程中无需使用源数据,通常在测试阶段对预训练深度模型进行自适应调整。然而,在医学图像分割的真实临床场景中,训练好的模型通常在测试阶段保持冻结。本文提出面向医学图像分割SFUDA的傅里叶视觉提示(FVP)。受自然语言处理中提示学习的启发,FVP通过向输入目标数据添加视觉提示来引导冻结的预训练模型在目标域中表现良好。在FVP中,视觉提示仅通过输入频率空间中少量低频可学习参数进行参数化,并通过最小化冻结模型下提示目标图像的预测分割与目标图像可靠伪分割标签之间的分割损失来学习。据我们所知,FVP是首个将视觉提示应用于医学图像分割SFUDA的工作。所提出的FVP在三个公开数据集上进行了验证,实验表明,与现有多种方法相比,FVP能产生更优的分割结果。