Learning style refers to a type of training mechanism adopted by an individual to gain new knowledge. As suggested by the VARK model, humans have different learning preferences, like Visual (V), Auditory (A), Read/Write (R), and Kinesthetic (K), for acquiring and effectively processing information. Our work endeavors to leverage this concept of knowledge diversification to improve the performance of model compression techniques like Knowledge Distillation (KD) and Mutual Learning (ML). Consequently, we use a single-teacher and two-student network in a unified framework that not only allows for the transfer of knowledge from teacher to students (KD) but also encourages collaborative learning between students (ML). Unlike the conventional approach, where the teacher shares the same knowledge in the form of predictions or feature representations with the student network, our proposed approach employs a more diversified strategy by training one student with predictions and the other with feature maps from the teacher. We further extend this knowledge diversification by facilitating the exchange of predictions and feature maps between the two student networks, enriching their learning experiences. We have conducted comprehensive experiments with three benchmark datasets for both classification and segmentation tasks using two different network architecture combinations. These experimental results demonstrate that knowledge diversification in a combined KD and ML framework outperforms conventional KD or ML techniques (with similar network configuration) that only use predictions with an average improvement of 2%. Furthermore, consistent improvement in performance across different tasks, with various network architectures, and over state-of-the-art techniques establishes the robustness and generalizability of the proposed model

翻译：学习风格指个体获取新知识所采用的一种训练方式。正如VARK模型所提出的，人类在获取和有效处理信息时具有不同的学习偏好，如视觉型（V）、听觉型（A）、读写型（R）和动觉型（K）。我们的工作致力于利用这种知识多样化的概念，来提升知识蒸馏（KD）和互学习（ML）等模型压缩技术的性能。为此，我们采用了一个包含单一教师网络和两个学生网络的统一框架，该框架不仅允许知识从教师向学生转移（KD），还鼓励学生之间的协作学习（ML）。与传统的教师以预测结果或特征表示形式与学生网络共享相同知识的方法不同，我们提出的方法采用了一种更趋于多样化的策略：训练一个学生使用教师的预测结果，而另一个学生使用教师的特征图。我们进一步扩展这种知识多样化，通过促进学生网络之间交换预测结果和特征图，丰富它们的学习体验。我们使用两种不同的网络架构组合，在三个基准数据集上针对分类和分割任务进行了全面的实验。这些实验结果表明，在KD和ML的结合框架中引入知识多样化，其性能优于仅使用预测结果的传统KD或ML技术（采用相似的网络配置），平均提升约2%。此外，在不同任务、不同网络架构以及相对于现有先进技术中，性能均保持一致的提升，这验证了所提出模型的鲁棒性和泛化能力。