SNNs promise energy-efficient and low-latency inference, but their performance still trails that of ANNs. ANN-to-SNN knowledge distillation helps narrow this gap, yet the original training data are often unavailable in practical deployment settings. Existing data-free knowledge distillation (DFKD) methods synthesize surrogate data by matching teacher-side priors, especially BN statistics, but these ANN-oriented constraints mainly regularize mean and variance and therefore remain under-constrained for SNN students whose responses depend on threshold-crossing dynamics. In this paper, we propose Spike Tail-Aware Relational Synthesis (STARS), a plug-and-play method for ANN-to-SNN DFKD that augments standard BN-guided synthesis with two complementary objectives: Relational Consistency Alignment, which preserves cross-sample relational consistency between teacher and student, and Tail-Aware Regularization, which regularizes threshold-relevant tail probabilities through soft exceedance over teacher-derived thresholds. Together, these objectives generate synthetic batches that remain teacher-valid while becoming more informative for SNN students. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet across multiple ANN-SNN pairs show that our method consistently improves conventional DFKD baselines and even surpasses several KD methods, with gains of up to 4.6\% on CIFAR-10 and 6.7\% on CIFAR-100, highlighting the importance of complementing BN matching with relational and tail-aware constraints in SNN-oriented DFKD.
翻译:SNN具有低能耗和低延迟推理的潜力,但其性能仍落后于ANN。ANN到SNN的知识蒸馏有助于缩小这一差距,然而在实际部署场景中原始训练数据往往不可用。现有无数据知识蒸馏方法通过匹配教师侧先验(尤其是BN统计量)来合成替代数据,但这些面向ANN的约束主要正则化均值和方差,因此对响应依赖于跨阈值动力学的SNN学生而言约束不足。本文提出脉冲尾感知关系合成方法——一种即插即用的ANN到SNN无数据知识蒸馏方法,通过两个互补目标增强标准BN引导合成:关系一致性对齐(保留教师与学生间的跨样本关系一致性)和尾感知正则化(通过软超越教师派生阈值来正则化阈值相关尾概率)。这些目标共同生成的合成批次既能保持教师有效性,又对SNN学生更具信息量。在CIFAR-10、CIFAR-100和Tiny-ImageNet上跨多个ANN-SNN对的实验表明,该方法持续改进传统无数据知识蒸馏基线,甚至超越若干知识蒸馏方法,在CIFAR-10和CIFAR-100上分别获得最高4.6%和6.7%的性能提升,凸显了在面向SNN的无数据知识蒸馏中补充BN匹配以关系与尾感知约束的重要性。