Recently, foundation models have been introduced demonstrating various tasks in the field of computer vision. These models such as Segment Anything Model (SAM) are generalized models trained using huge datasets. Currently, ongoing research focuses on exploring the effective utilization of these generalized models for specific domains, such as medical imaging. However, in medical imaging, the lack of training samples due to privacy concerns and other factors presents a major challenge for applying these generalized models to medical image segmentation task. To address this issue, the effective fine tuning of these models is crucial to ensure their optimal utilization. In this study, we propose to combine a complementary Convolutional Neural Network (CNN) along with the standard SAM network for medical image segmentation. To reduce the burden of fine tuning large foundation model and implement cost-efficient trainnig scheme, we focus only on fine-tuning the additional CNN network and SAM decoder part. This strategy significantly reduces trainnig time and achieves competitive results on publicly available dataset. The code is available at https://github.com/11yxk/SAM-LST.

翻译：近期，基础模型在计算机视觉领域展现出处理多种任务的能力。这些模型（如Segment Anything Model，SAM）是通过大规模数据集训练得到的通用模型。目前，研究重点在于探索如何将这些通用模型有效应用于特定领域，例如医学影像。然而，在医学影像中，由于隐私等因素导致训练样本匮乏，这为通用模型应用于医学图像分割任务带来了重大挑战。为解决这一问题，对这些模型进行有效的微调对于确保其最优利用至关重要。本研究提出将互补卷积神经网络（CNN）与标准SAM网络相结合，用于医学图像分割。为减轻大规模基础模型的微调负担并实现高效训练策略，我们仅对附加的CNN网络和SAM解码器部分进行微调。该策略显著缩短了训练时间，并在公开数据集上取得了具有竞争力的结果。代码已开源至https://github.com/11yxk/SAM-LST。