Knowledge distillation (KD) has traditionally relied on a static teacher-student framework, where a large, well-trained teacher transfers knowledge to a single student model. However, these approaches often suffer from knowledge degradation, inefficient supervision, and reliance on either a very strong teacher model or large labeled datasets, which limits their effectiveness in real-world, limited-data scenarios. To address these, we present the first-ever Weakly-supervised Chain-based KD network (WeCKD) that redefines knowledge transfer through a structured sequence of interconnected models. Unlike conventional KD, it forms a progressive distillation chain, where each model not only learns from its predecessor but also refines the knowledge before passing it forward. This structured knowledge transfer further enhances feature learning, reduces data dependency, and mitigates the limitations of one-step KD. Each model in the distillation chain is trained on only a fraction of the dataset and demonstrates that effective learning can be achieved with minimal supervision. Extensive evaluations across four otoscopic imaging datasets demonstrate that it not only matches but in many cases surpasses the performance of existing supervised methods. Experimental results on two other datasets further underscore its generalization across diverse medical imaging modalities, including microscopic and magnetic resonance imaging. Furthermore, our evaluations resulted in cumulative accuracy gains of up to +23% over a single backbone trained on the same limited data, which highlights its potential for real-world adoption.

翻译：知识蒸馏（KD）传统上依赖于静态的师生框架，即一个训练完备的大型教师模型将知识迁移至单个学生模型。然而，这类方法常面临知识退化、监督效率低下以及对强教师模型或大规模标注数据集的依赖等问题，限制了其在现实世界有限数据场景中的有效性。为解决这些问题，我们首次提出弱监督链式蒸馏网络（WeCKD），通过结构化的互联模型序列重新定义知识迁移机制。与常规KD不同，该方法构建了一个渐进式蒸馏链，其中每个模型不仅从其前驱模型学习知识，还会在向前传递前对知识进行精炼。这种结构化的知识迁移进一步增强了特征学习能力，降低了对数据的依赖，并缓解了一步式KD的局限性。蒸馏链中的每个模型仅使用数据集的子集进行训练，证明了在极少量监督下即可实现有效学习。在四个耳镜成像数据集上的广泛评估表明，该方法不仅能够匹配现有监督方法的性能，而且在多数情况下表现更优。在另外两个数据集上的实验结果进一步验证了其在不同医学成像模态（包括显微成像与磁共振成像）间的泛化能力。此外，我们的评估结果显示，相较于在相同有限数据上训练的单一骨干网络，该方法累计准确率提升最高可达+23%，凸显了其在现实场景中的应用潜力。