In this study, our goal is to show the impact of self-supervised pre-training of transformers for organ at risk (OAR) and tumor segmentation as compared to costly fully-supervised learning. The proposed algorithm is called Monte Carlo Transformer based U-Net (MC-Swin-U). Unlike many other available models, our approach presents uncertainty quantification with Monte Carlo dropout strategy while generating its voxel-wise prediction. We test and validate the proposed model on both public and one private datasets and evaluate the gross tumor volume (GTV) as well as nearby risky organs' boundaries. We show that self-supervised pre-training approach improves the segmentation scores significantly while providing additional benefits for avoiding large-scale annotation costs.

翻译：本研究旨在展示变压器模型自监督预训练在危及器官（OAR）与肿瘤分割任务中的效果，并与成本高昂的全监督学习进行对比。所提出的算法名为基于蒙特卡洛变压器的U-Net（MC-Swin-U）。与现有众多模型不同，本方法在生成体素级预测时，采用蒙特卡洛丢弃策略实现不确定性量化。我们在一个公共数据集和一个私有数据集上对模型进行测试与验证，并评估大体肿瘤体积（GTV）及邻近危险器官边界的分割效果。结果表明，自监督预训练方法显著提升了分割评分，同时为避免大规模标注成本提供了额外优势。