Driven by the latest trend towards self-supervised learning (SSL), the paradigm of "pretraining-then-finetuning" has been extensively explored to enhance the performance of clinical applications with limited annotations. Previous literature on model finetuning has mainly focused on regularization terms and specific policy models, while the misalignment of channels between source and target models has not received sufficient attention. In this work, we revisited the dynamics of batch normalization (BN) layers and observed that the trainable affine parameters of BN serve as sensitive indicators of domain information. Therefore, Affine Collaborative Normalization (AC-Norm) is proposed for finetuning, which dynamically recalibrates the channels in the target model according to the cross-domain channel-wise correlations without adding extra parameters. Based on a single-step backpropagation, AC-Norm can also be utilized to measure the transferability of pretrained models. We evaluated AC-Norm against the vanilla finetuning and state-of-the-art fine-tuning methods on transferring diverse pretrained models to the diabetic retinopathy grade classification, retinal vessel segmentation, CT lung nodule segmentation/classification, CT liver-tumor segmentation and MRI cardiac segmentation tasks. Extensive experiments demonstrate that AC-Norm unanimously outperforms the vanilla finetuning by up to 4% improvement, even under significant domain shifts where the state-of-the-art methods bring no gains. We also prove the capability of AC-Norm in fast transferability estimation. Our code is available at https://github.com/EndoluminalSurgicalVision-IMR/ACNorm.

翻译：受自监督学习最新趋势驱动，“预训练-微调”范式已被广泛探索以增强标注有限的临床应用性能。先前的模型微调研究主要关注正则化项和特定策略模型，而源模型与目标模型之间的通道错位问题未得到充分关注。本研究重新审视了批归一化层的动态特性，发现BN的可训练仿射参数可作为领域信息的敏感指标。据此，我们提出仿射协同归一化用于微调，该方法无需增加额外参数即可根据跨域通道相关性动态校准目标模型的通道。基于单步反向传播，AC-Norm还可用于测量预训练模型的可迁移性。我们对比了AC-Norm与基础微调方法及当前最优微调方法在眼底病变分级、视网膜血管分割、CT肺结节分割/分类、CT肝脏肿瘤分割及MRI心脏分割任务中的表现。大量实验表明，即使在当前最优方法无法提升性能的显著领域偏移场景下，AC-Norm仍能统一超越基础微调方法，最高提升达4%。我们还验证了AC-Norm在快速迁移性评估中的能力。代码已开源：https://github.com/EndoluminalSurgicalVision-IMR/ACNorm。