There is no doubt that advanced artificial intelligence models and high quality data are the keys to success in developing computational pathology tools. Although the overall volume of pathology data keeps increasing, a lack of quality data is a common issue when it comes to a specific task due to several reasons including privacy and ethical issues with patient data. In this work, we propose to exploit knowledge distillation, i.e., utilize the existing model to learn a new, target model, to overcome such issues in computational pathology. Specifically, we employ a student-teacher framework to learn a target model from a pre-trained, teacher model without direct access to source data and distill relevant knowledge via momentum contrastive learning with multi-head attention mechanism, which provides consistent and context-aware feature representations. This enables the target model to assimilate informative representations of the teacher model while seamlessly adapting to the unique nuances of the target data. The proposed method is rigorously evaluated across different scenarios where the teacher model was trained on the same, relevant, and irrelevant classification tasks with the target model. Experimental results demonstrate the accuracy and robustness of our approach in transferring knowledge to different domains and tasks, outperforming other related methods. Moreover, the results provide a guideline on the learning strategy for different types of tasks and scenarios in computational pathology. Code is available at: \url{https://github.com/trinhvg/MoMA}.

翻译：毫无疑问，先进的人工智能模型与高质量数据是开发计算病理学工具成功的关键。尽管病理学数据总量持续增长，但由于患者数据涉及隐私和伦理等问题，特定任务中普遍存在缺乏高质量数据的现象。本文提出利用知识蒸馏（即利用现有模型学习新的目标模型）来克服计算病理学中的此类问题。具体而言，我们采用学生-教师框架，通过动量对比学习结合多头注意力机制，在无需直接访问源数据的情况下，从预训练的教师模型中学习目标模型，从而获得一致且具备上下文感知能力的特征表征。这使得目标模型能够吸收教师模型的信息表征，同时无缝适应目标数据的独特细节。该方法在教师模型与目标模型分别基于相同、相关及不相关分类任务训练的多种场景下进行了严格评估。实验结果表明，本方法在跨领域及跨任务的知识迁移中展现出准确性与鲁棒性，性能优于其他相关方法。此外，研究结果为计算病理学中不同类型任务与场景的学习策略提供了指导。代码开源地址：\url{https://github.com/trinhvg/MoMA}。