In medical image analysis, transfer learning is a powerful method for deep neural networks (DNNs) to generalize well on limited medical data. Prior efforts have focused on developing pre-training algorithms on domains such as lung ultrasound, chest X-ray, and liver CT to bridge domain gaps. However, we find that model fine-tuning also plays a crucial role in adapting medical knowledge to target tasks. The common fine-tuning method is manually picking transferable layers (e.g., the last few layers) to update, which is labor-expensive. In this work, we propose a meta-learning-based LR tuner, named MetaLR, to make different layers automatically co-adapt to downstream tasks based on their transferabilities across domains. MetaLR learns appropriate LRs for different layers in an online manner, preventing highly transferable layers from forgetting their medical representation abilities and driving less transferable layers to adapt actively to new domains. Extensive experiments on various medical applications show that MetaLR outperforms previous state-of-the-art (SOTA) fine-tuning strategies. Codes are released.

翻译：摘要：在医学图像分析中，迁移学习是深度神经网络在有限医学数据上实现良好泛化能力的有效方法。以往研究主要致力于开发肺超声、胸部X光片及肝脏CT等领域的预训练算法以弥合领域差异。然而，我们发现模型微调在适配医学知识至目标任务中同样具有关键作用。常见的微调方法需人工选择可迁移层（如最后若干层）进行更新，这需要大量人工成本。本研究提出基于元学习的学习率调优器MetaLR，使不同层能根据其跨领域迁移能力自动协同适应下游任务。MetaLR以在线方式为各层学习适宜的学习率，既能防止高迁移层丧失医学表征能力，又能驱动低迁移层主动适应新领域。在多个医学应用场景的广泛实验表明，MetaLR的性能优于现有最先进的微调策略。相关代码已开源。