Multi-modality medical imaging is crucial in clinical treatment as it can provide complementary information for medical image segmentation. However, collecting multi-modal data in clinical is difficult due to the limitation of the scan time and other clinical situations. As such, it is clinically meaningful to develop an image segmentation paradigm to handle this missing modality problem. In this paper, we propose a prototype knowledge distillation (ProtoKD) method to tackle the challenging problem, especially for the toughest scenario when only single modal data can be accessed. Specifically, our ProtoKD can not only distillate the pixel-wise knowledge of multi-modality data to single-modality data but also transfer intra-class and inter-class feature variations, such that the student model could learn more robust feature representation from the teacher model and inference with only one single modality data. Our method achieves state-of-the-art performance on BraTS benchmark.

翻译：多模态医学成像在临床治疗中至关重要，因为它能够为医学图像分割提供互补信息。然而，由于扫描时间限制及其他临床状况，临床上收集多模态数据存在困难。因此，开发一种能够处理缺失模态问题的图像分割范式具有重要的临床意义。本文提出了一种原型知识蒸馏（ProtoKD）方法来应对这一挑战性问题，特别是在仅能获取单模态数据的最困难场景下。具体而言，我们的ProtoKD不仅能够将多模态数据的像素级知识蒸馏到单模态数据中，还能传递类内和类间的特征变化，从而使学生模型能够从教师模型中学习到更鲁棒的特征表示，并仅通过单一模态数据进行推理。我们的方法在BraTS基准测试上达到了最先进的性能。