Even though deep speaker models have demonstrated impressive accuracy in speaker verification tasks, this often comes at the expense of increased model size and computation time, presenting challenges for deployment in resource-constrained environments. Our research focuses on addressing this limitation through the development of small footprint deep speaker embedding extraction using knowledge distillation. While previous work in this domain has concentrated on speaker embedding extraction at the utterance level, our approach involves amalgamating embeddings from different levels of the x-vector model (teacher network) to train a compact student network. The results highlight the significance of frame-level information, with the student models exhibiting a remarkable size reduction of 85%-91% compared to their teacher counterparts, depending on the size of the teacher embeddings. Notably, by concatenating teacher embeddings, we achieve student networks that maintain comparable performance to the teacher while enjoying a substantial 75% reduction in model size. These findings and insights extend to other x-vector variants, underscoring the broad applicability of our approach.

翻译：尽管深度说话人模型在声纹验证任务中展现出卓越的准确率，但这往往以模型规模扩大和计算时间增加为代价，给资源受限环境下的部署带来挑战。本研究聚焦于通过知识蒸馏开发小规模深度说话人嵌入提取技术以弥补这一局限。不同于以往该领域工作集中于话语级别的说话人嵌入提取，我们的方法通过融合x-vector模型（教师网络）不同层级的嵌入来训练紧凑的学生网络。实验结果表明帧级信息具有显著重要性，学生模型相较于教师模型实现了85%-91%的规模缩减，具体压缩比例取决于教师嵌入的维度。值得注意的是，通过拼接教师嵌入，我们获得的学生网络在保持与教师相当性能的同时，模型规模实现了75%的大幅缩减。这些发现和结论可推广至其他x-vector变体，充分印证了该方法具有广泛适用性。