Harnessing the power of pre-training on large-scale datasets like ImageNet forms a fundamental building block for the progress of representation learning-driven solutions in computer vision. Medical images are inherently different from natural images as they are acquired in the form of many modalities (CT, MR, PET, Ultrasound etc.) and contain granulated information like tissue, lesion, organs etc. These characteristics of medical images require special attention towards learning features representative of local context. In this work, we focus on designing an effective pre-training framework for 3D radiology images. First, we propose a new masking strategy called local masking where the masking is performed across channel embeddings instead of tokens to improve the learning of local feature representations. We combine this with classical low-level perturbations like adding noise and downsampling to further enable low-level representation learning. To this end, we introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. Additionally, we also devise a cross-modal contrastive loss (CMCL) to accommodate the pre-training of multiple modalities in a single framework. We curate a large-scale dataset to enable pre-training of 3D medical radiology images (MRI and CT). The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance. Notably, our proposed method tops the public test leaderboard of BTCV multi-organ segmentation challenge.

翻译：利用ImageNet等大规模数据集进行预训练，是推动计算机视觉中表征学习驱动解决方案进步的基础环节。医学图像与自然图像存在本质差异，其获取方式涉及多种模态（CT、MR、PET、超声等），且包含组织、病灶、器官等颗粒化信息。这些特性要求在学习局部上下文特征时给予特别关注。本研究聚焦于为三维放射影像设计有效的预训练框架。首先，我们提出一种名为局部掩码的新策略，该策略通过对通道嵌入而非令牌进行掩码操作，以提升局部特征表征的学习能力。我们将此与添加噪声、降采样等经典低层扰动方法相结合，进一步促进低层表征学习。基于此，我们引入破坏性自编码器这一预训练框架，其通过局部掩码与低层扰动组合生成的破坏信号，尝试重建原始图像。此外，我们还设计了一种跨模态对比损失函数，使单个框架能够支持多模态预训练。通过构建大规模数据集，我们实现了对三维医学放射影像（MRI和CT）的预训练。所提出的预训练框架在多个下游任务中进行了测试，均取得了最优性能。值得注意的是，我们的方法在BTCV多器官分割挑战赛公开测试排行榜上位居首位。