Transformer-based language models of code have achieved state-of-the-art performance across a wide range of software analytics tasks, but their practical deployment remains limited due to high computational costs, slow inference speeds, and significant environmental impact. To address these challenges, recent research has increasingly explored knowledge distillation as a method for compressing a large language model of code (the teacher) into a smaller model (the student) while maintaining performance. However, the degree to which a student model deeply mimics the predictive behavior and internal representations of its teacher remains largely unexplored, as current accuracy-based evaluation provides only a surface-level view of model quality and often fails to capture more profound discrepancies in behavioral fidelity between the teacher and student models. To address this gap, we empirically show that the student model often fails to deeply mimic the teacher model, resulting in up to 285% greater performance drop under adversarial attacks, which is not captured by traditional accuracy-based evaluation. Therefore, we propose MetaCompress, a metamorphic testing framework that systematically evaluates behavioral fidelity by comparing the outputs of teacher and student models under a set of behavior-preserving metamorphic relations. We evaluate MetaCompress on two widely studied tasks, using compressed versions of popular language models of code, obtained via three different knowledge distillation techniques: Compressor, AVATAR, and MORPH. The results show that MetaCompress identifies up to 62% behavioral discrepancies in student models, underscoring the need for behavioral fidelity evaluation within the knowledge distillation pipeline and establishing MetaCompress as a practical framework for testing compressed language models of code derived through knowledge distillation.

翻译：基于Transformer的代码语言模型在各类软件分析任务中均取得了最先进的性能，但其实际部署仍因高计算成本、缓慢推理速度和显著环境影响而受限。为应对这些挑战，近年来研究逐渐探索将知识蒸馏作为压缩大型代码语言模型（教师模型）为小型模型（学生模型）的方法，同时保持模型性能。然而，学生模型在多大程度上深度模仿了教师模型的预测行为和内部表征仍鲜有探讨——当前基于准确率的评估仅提供模型质量的表层视角，往往无法捕捉教师与学生模型间行为保真度的深层差异。为弥补这一空白，我们通过实验证明：学生模型常未能深度模仿教师模型，在对抗性攻击下其性能下降幅度高达285%，但传统准确率评估无法捕捉这一现象。为此，我们提出MetaCompress——一个通过比较教师与学生模型在行为保留蜕变关系下的输出来系统性评估行为保真度的蜕变测试框架。我们基于两种广泛研究的任务，采用三种不同知识蒸馏技术（Compressor、AVATAR和MORPH）获得的压缩版流行代码语言模型来评估MetaCompress。结果表明，MetaCompress能识别出学生模型中高达62%的行为差异，这凸显了在知识蒸馏流程中纳入行为保真度评估的必要性，并使MetaCompress成为测试由知识蒸馏生成的压缩代码语言模型的实用框架。