Efficient medical image segmentation aims to provide accurate pixel-wise predictions for medical images with a lightweight implementation framework. However, lightweight frameworks generally fail to achieve superior performance and suffer from poor generalizable ability on cross-domain tasks. In this paper, we explore the generalizable knowledge distillation for the efficient segmentation of cross-domain medical images. Considering the domain gaps between different medical datasets, we propose the Model-Specific Alignment Networks (MSAN) to obtain the domain-invariant representations. Meanwhile, a customized Alignment Consistency Training (ACT) strategy is designed to promote the MSAN training. Considering the domain-invariant representative vectors in MSAN, we propose two generalizable knowledge distillation schemes for cross-domain distillation, Dual Contrastive Graph Distillation (DCGD) and Domain-Invariant Cross Distillation (DICD). Specifically, in DCGD, two types of implicit contrastive graphs are designed to represent the intra-coupling and inter-coupling semantic correlations from the perspective of data distribution. In DICD, the domain-invariant semantic vectors from the two models (i.e., teacher and student) are leveraged to cross-reconstruct features by the header exchange of MSAN, which achieves improvement in the generalization of both the encoder and decoder in the student model. Furthermore, a metric named Frechet Semantic Distance (FSD) is tailored to verify the effectiveness of the regularized domain-invariant features. Extensive experiments conducted on the Liver and Retinal Vessel Segmentation datasets demonstrate the superiority of our method, in terms of performance and generalization on lightweight frameworks.

翻译：高效医学图像分割旨在通过轻量级实现框架为医学图像提供精确的逐像素预测。然而，轻量级框架通常难以取得优异性能，且在跨领域任务中泛化能力不足。本文探索了面向跨领域医学图像高效分割的可泛化知识蒸馏方法。针对不同医学数据集之间的领域差异，我们提出模型特定对齐网络（MSAN）以获取域不变表示；同时设计了定制化的对齐一致性训练（ACT）策略来优化MSAN的训练过程。基于MSAN中的域不变表征向量，我们提出两种可泛化的跨领域知识蒸馏方案：双对比图蒸馏（DCGD）与域不变交叉蒸馏（DICD）。具体而言，DCGD通过构建两种隐式对比图，从数据分布角度表征类内与类间语义相关性；DICD则利用教师模型与学生模型中的域不变语义向量，通过MSAN的头部交换机制实现特征的交叉重构，从而提升学生模型中编码器与解码器的泛化能力。此外，我们专门设计了弗雷歇语义距离（FSD）指标来验证正则化域不变特征的有效性。在肝脏与视网膜血管分割数据集上的大量实验表明，本方法在轻量级框架的性能与泛化性方面均具有显著优势。