We propose ClassroomKD, a novel multi-mentor knowledge distillation framework inspired by classroom environments to enhance knowledge transfer between student and multiple mentors. Unlike traditional methods that rely on fixed mentor-student relationships, our framework dynamically selects and adapts the teaching strategies of diverse mentors based on their effectiveness for each data sample. ClassroomKD comprises two main modules: the Knowledge Filtering (KF) Module and the Mentoring Module. The KF Module dynamically ranks mentors based on their performance for each input, activating only high-quality mentors to minimize error accumulation and prevent information loss. The Mentoring Module adjusts the distillation strategy by tuning each mentor's influence according to the performance gap between the student and mentors, effectively modulating the learning pace. Extensive experiments on image classification (CIFAR-100 and ImageNet) and 2D human pose estimation (COCO Keypoints and MPII Human Pose) demonstrate that ClassroomKD significantly outperforms existing knowledge distillation methods. Our results highlight that a dynamic and adaptive approach to mentor selection and guidance leads to more effective knowledge transfer, paving the way for enhanced model performance through distillation.

翻译：我们提出ClassroomKD，一种受课堂环境启发的新型多导师知识蒸馏框架，旨在增强学生与多位导师之间的知识迁移。与传统方法依赖固定的师生关系不同，我们的框架根据每位导师对每个数据样本的有效性，动态选择并调整不同导师的教学策略。ClassroomKD包含两个主要模块：知识过滤模块和指导模块。知识过滤模块根据每位导师对每个输入的表现动态排序，仅激活高质量导师以最小化误差累积并防止信息丢失。指导模块通过根据学生与导师之间的表现差距调整每位导师的影响力，从而调整蒸馏策略，有效调节学习节奏。在图像分类和二维人体姿态估计上的大量实验表明，ClassroomKD显著优于现有知识蒸馏方法。我们的结果突显了动态自适应的导师选择与指导方法能实现更有效的知识迁移，为通过蒸馏提升模型性能开辟了新途径。