There is no doubt that advanced artificial intelligence models and high quality data are the keys to success in developing computational pathology tools. Although the overall volume of pathology data keeps increasing, a lack of quality data is a common issue when it comes to a specific task due to several reasons including privacy and ethical issues with patient data. In this work, we propose to exploit knowledge distillation, i.e., utilize the existing model to learn a new, target model, to overcome such issues in computational pathology. Specifically, we employ a student-teacher framework to learn a target model from a pre-trained, teacher model without direct access to source data and distill relevant knowledge via momentum contrastive learning with multi-head attention mechanism, which provides consistent and context-aware feature representations. This enables the target model to assimilate informative representations of the teacher model while seamlessly adapting to the unique nuances of the target data. The proposed method is rigorously evaluated across different scenarios where the teacher model was trained on the same, relevant, and irrelevant classification tasks with the target model. Experimental results demonstrate the accuracy and robustness of our approach in transferring knowledge to different domains and tasks, outperforming other related methods. Moreover, the results provide a guideline on the learning strategy for different types of tasks and scenarios in computational pathology. Code is available at: \url{https://github.com/trinhvg/MoMA}.

翻译：毫无疑问，先进的人工智能模型与高质量数据是开发计算病理学工具取得成功的关键。尽管病理学数据总量持续增长，但由于患者数据隐私及伦理问题等诸多原因，针对特定任务时缺乏高质量数据仍是普遍存在的难题。本研究提出通过知识蒸馏（即利用现有模型学习新的目标模型）来解决计算病理学领域的此类问题。具体而言，我们采用师生框架，在不直接访问源数据的情况下从预训练的教师模型中学习目标模型，并通过结合多头注意力机制的动量对比学习来提取相关知识，该机制能够提供一致且上下文感知的特征表示。这使得目标模型既能吸收教师模型的信息表征，又能无缝适应目标数据特有的细微差异。我们在教师模型与目标模型接受相同任务、相关任务及无关任务训练的不同场景下对所提方法进行了严格评估。实验结果表明，该方法在跨域跨任务知识迁移中具有卓越的准确性与鲁棒性，性能优于其他相关方法。此外，实验结果还为计算病理学中不同类型任务与场景的学习策略提供了指导准则。代码发布于：\url{https://github.com/trinhvg/MoMA}。