Medical image segmentation methods normally perform poorly when there is a domain shift between training and testing data. Unsupervised Domain Adaptation (UDA) addresses the domain shift problem by training the model using both labeled data from the source domain and unlabeled data from the target domain. Source-Free UDA (SFUDA) was recently proposed for UDA without requiring the source data during the adaptation, due to data privacy or data transmission issues, which normally adapts the pre-trained deep model in the testing stage. However, in real clinical scenarios of medical image segmentation, the trained model is normally frozen in the testing stage. In this paper, we propose Fourier Visual Prompting (FVP) for SFUDA of medical image segmentation. Inspired by prompting learning in natural language processing, FVP steers the frozen pre-trained model to perform well in the target domain by adding a visual prompt to the input target data. In FVP, the visual prompt is parameterized using only a small amount of low-frequency learnable parameters in the input frequency space, and is learned by minimizing the segmentation loss between the predicted segmentation of the prompted target image and reliable pseudo segmentation label of the target image under the frozen model. To our knowledge, FVP is the first work to apply visual prompts to SFUDA for medical image segmentation. The proposed FVP is validated using three public datasets, and experiments demonstrate that FVP yields better segmentation results, compared with various existing methods.
翻译:医学图像分割方法在训练数据与测试数据存在域偏移时通常表现不佳。无监督域适应通过利用源域有标签数据和目标域无标签数据共同训练模型来解决域偏移问题。近期提出的无源无监督域适应方法因数据隐私或传输限制,无需在适应过程中使用源数据,通常在测试阶段对预训练深度模型进行微调。然而,在真实医学图像分割临床场景中,训练完成的模型在测试阶段通常保持冻结状态。本文提出傅里叶视觉提示用于医学图像分割的无源无监督域适应。受自然语言处理中提示学习启发,FVP通过向目标输入数据添加视觉提示,引导冻结的预训练模型在目标域实现良好性能。在FVP中,视觉提示仅利用输入频率空间中少量低频可学习参数进行参数化,并通过最小化冻结模型下提示后目标图像预测分割与目标图像可靠伪分割标签之间的分割损失来学习。据我们所知,FVP是首个将视觉提示应用于医学图像分割无源无监督域适应的工作。所提出的FVP在三个公开数据集上得到验证,实验表明,与多种现有方法相比,FVP能获得更优的分割结果。