Instance-aware embeddings predicted by deep neural networks have revolutionized biomedical instance segmentation, but its resource requirements are substantial. Knowledge distillation offers a solution by transferring distilled knowledge from heavy teacher networks to lightweight yet high-performance student networks. However, existing knowledge distillation methods struggle to extract knowledge for distinguishing instances and overlook global relation information. To address these challenges, we propose a graph relation distillation approach for efficient biomedical instance segmentation, which considers three essential types of knowledge: instance-level features, instance relations, and pixel-level boundaries. We introduce two graph distillation schemes deployed at both the intra-image level and the inter-image level: instance graph distillation (IGD) and affinity graph distillation (AGD). IGD constructs a graph representing instance features and relations, transferring these two types of knowledge by enforcing instance graph consistency. AGD constructs an affinity graph representing pixel relations to capture structured knowledge of instance boundaries, transferring boundary-related knowledge by ensuring pixel affinity consistency. Experimental results on a number of biomedical datasets validate the effectiveness of our approach, enabling student models with less than $ 1\%$ parameters and less than $10\%$ inference time while achieving promising performance compared to teacher models.

翻译：深度神经网络预测的实例感知嵌入已革新了生物医学实例分割，但其资源需求巨大。知识蒸馏通过将蒸馏知识从繁重的教师网络迁移至轻量级高性能学生网络提供了解决方案。然而，现有知识蒸馏方法难以提取用于区分实例的知识，并忽略了全局关系信息。为应对这些挑战，我们提出了一种面向高效生物医学实例分割的图关系蒸馏方法，该方法考虑了三种核心知识：实例级特征、实例关系及像素级边界。我们引入了两种部署于图像内与图像间层级上的图蒸馏方案：实例图蒸馏（IGD）与亲和力图蒸馏（AGD）。IGD构建了表征实例特征与关系的图，通过强制实例图一致性来迁移这两类知识。AGD构建了表征像素关系的亲和力图以捕获实例边界的结构化知识，通过确保像素亲和力一致性来迁移边界相关知识。在多个生物医学数据集上的实验结果验证了本方法的有效性，使学生模型在参数不足教师模型的$1\%$、推理时间不足教师模型的$10\%$的情况下，仍能取得与教师模型相当的性能。